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(54) Image pickup apparatus and method for composing images 

(57) By using the image pickup apparatus and the 
method for processing images obtained by means of 
same according to the present invention it is possible to 
re^ster a plurality of mask patterns, to select desired 
one among the plurality of registered mask patterns, 
and to cut-out a desired part of a taken image by using 
the desired mask pattern to synthesize a new image, 
which is stored. In addition, it is possisle also to extract 
a part of image or the whole image of a stored taken 
image to be registered as a mask pattern. 
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Description 

This Invention relates to an image pidoip apparatus for imaging a subject and an image processing method, and in 
particular, it is suitiable for a digital still camera Ixit is not restricted within such device. 
5 Recently a digital camera has been developed, in which an image of an object transformed into electric signals by 

means of imaging elements such as CCD, etc. is converted into digital data (hereint>elow called image data) to t>e 
stored in a recording medium, which image can be reproduced according to desire of a user. Such cameras are spread- 
ing in a market as image recording/reproducing apparatuses and image inputting apparatuses fbr personal computers, 
etc. Further most of them are provided with a display device for displaying an image of an object at imaging arKi at the 
10 same time for displaying a reproduced image at reproduction. 

In order to process video image data stored in a digital camera, the video image data is transmitted to an informa- 
tion processing apparatus such as a personal conrputer. etc., stored therein and the stored video image data is proc- 
essed with an application program fbr video image processing. 

However, in a case that a video image data is processed with a conventional application program for video image 
15 processing, a processing operation but it is very complicated, and for example, it can not be done easily to clip a desired 
part of a video image and to compose another video image it. 

An object of the invention is to an imaging apparatus and a video image processing method suitable for clipping a 
desired part of a video image, and fbr composing another video image the dipped video image. 

In order to attain the object of the invention, an image pickup apparatus accordir)g to the invention which comprises 
20 an image pickup means fbr imaging a subject to output an image and Image memory means Ibr storing the image out- 
putted from said image pickup means, and is characterized by employment of pattern memory means for storing at 
least one pattern, clipped image producing means fbr clipping a corresponding part off the image stored in said image 
memory means with the pattern stored in said pattern memory means, and image composing means for composing a 
desired image the clipped image produced by said dipped image producing means. 
26 Further a image processing method according to the invention comprises a first step clipping a corresponding part 
of an image with a pattern and a second step composing a desired image the dipped image. 

The at>ove apparatus and metiiod of the invention will be explained in detail in the following. 

(Definition of terms used in the present specif k^ion) 

30 

In the following explanation of the present invention following terms between parentheses have following means. 

(mask pattern): a pattern used as a mask fbr processing of cutting-out (extracting) an object image; 
(cut-out image): a part of an image cut-out (extracted) from an object image by a mask processing; 
35 (through-image) : a monitor moving image of an object displayed on a screen of a display devk;e at imaging; 

(stored image): includes an image stored by imaging, an image prevkxisly registered, a monochrome Image; an 
image registered in a memory medium mountabte on arxf dismountable f rom an imaging apparatus such as an 10 
card, an ROM, etc.; 

(background image): a cut-out Image displayed on a display device or a stored irhage reproduced and displayed on 
40 a display device. 

(Summary) 

An imaging apparatus according to the present invention is provUed with a such auxiliary function of processing an 
4s inrtage that a part of a taken Image is combined with anottier Image to synthesize a new image. The auxiliary function 
of processing an image includes functions as indicated below in (a) to (h). 

(a) an image synthesizing furx:tion off cutting-out a part of a taken image by using a mask pattern with another 
image to synthesize a cut-out image; 
50 (b) a pattern production/registration function of extracting a mask pattern from a through-image displayed on a dis- 
play device at imaging to register (record) it on a recording medium as the mask pattern; 

(c) a cut-out image producting/registering f unctton off combining a mask pattern selected by a user among a plural- 
ity of registered mask patterns with another through-image to synthesize a new image at imaging and to display it 
on a display device and of registering (recording) the synthesized image selected by the user as a cut-out image 

55 on a recording medium; 

(d) an image sticking synthesizing function of sticking a cut-out image selected by a user among registered cut-out 
images witti anotiier through-image to synthesize a new image at imaging and to display it on a display device and 
off recording the synthesized image selected by the user on a recording medium; 
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(e) a pattern production/registration function of extracting a mask pattern from a recorded image recorded on a 
recording medium at recording to register it on a recording medium as a mask pattern; 

(f) a cut-out Image producting/registering function of combining a mask pattern selected by a user among a plurality 
of registered mask patterns with another through-image to synthesize a nea image at reproductbn and to display 

5 it on a display device and of registering (recording) the synthesized image selected by the user on a recofding 

medium; 

(g) an image sticking synthesizing function of sticking a cut-out image selected by a user among registered cut-out 
images with a reproduced stored image to synthesize a new image at reproduction and to display it on a display 
device and of recording the synthesized Image selected by the user on a recording medium; and 

10 (h) a guiding frame displaying function of displaying a guiding frame giving a criterk>n of magnitude of an image to 
be taken at imaging. 

Further the pattern production/registration functk>n can Include further fbllowing functions: 

IS (I) a function of detecting/extracting a contour of an image displayed on a display device at imaging or at reproduc- 
tion to register it as a mask pattern on a recording medium; 

G) a registering function of taking-in distances to a plurality of objects in a through-image to record them together 
with the image at imaging and designating a distance on a reproduced stored image to extract a mask pattern from 
an Image of objects, which are at a same distance, at reproduction to register it on a recording medium; and 
20 (i^ a function of drawing/displaying an innage by means of a pointing device such as a cursor or an inputting pen 
and e)ctracting a mask pattern thereof to register it on a recording medium. 

Moreover It can Include furttier following functions: 

25 (I) a preview display function of displaying at a high speed for displaying mask patterns, cut-out Images and back- 
ground images on a display device at selection; and 

(m) a function of automatically determining resolution fbr automatically determining resolution so that recording 
speed is highest at recording a synthesized image. 

$0 Fig. 1 is a t^ock diagram showing an example of construction of a digital camera, which is an embodiment of the 
Imaging apparatus according to the present invention; 

Fig. 2 is a diagram showing an example of construction of a key input section Indicated in Fig. 1 ; 

Fig. 3 is a diagram for explaining processing nrxxies of the digital camera indicated in Rg. 1; 

Figs. 4A to 4C are block diagrams showing an example of construction of image processing means; 
35 Figs. 5A to 5C are diagrams showing examples of a link table; 

Rgs. 6 A to 6C are diagrams fbr explaining an example of synthesis by combining an taken Innage with a cut-out 
. image;.. 

Rg. 7 Is a flow chart showing an embodiment of a pattern productiorVregistratron processing operation; 

Figs. 8A to 8C are diagrams for explaining a pattern production/^egisti'ation process, based on the flow chart irxJi- 
40 cated in Fig. 7; 

Fig. 9 is a flow chart showing an embodiment of a cut-out Image production/registration processing operation; 
Figs. 10A to 10D are diagrams for explaining a cut-out image production/registration process, based on the flow 
chart Indicated in Rg. 9; 

Rgs. 1 1 A to 1 1 D are diagrams fbr explaining another cut-out image production/registration process, based on the 
45 flow Chart irvlicated in Fig. 9; 

Rgs. 12A and 12B are flow charts showing an embodiment of a cutting-out image synthesis processing operation 
of the digital camera Indicated in Fig. 1 ; 

Figs. 13A to 13D are diagrams for explaining a out-out Image synthesizing process, based on the fkiw chart indi- 
cated in Rgs. 12A and 12B; 

60 Rgs. 14A and 14B are diagrams fbr explaining a cutting-out image synthesizing process, based on the flow chart 
indicated in Figs. 12Aand 12B; 

Rg. 15 is a flow chart showing an embodiment of a pattern production/registration processing operatkHi; 
Rgs. 1 6A to 1 6D are diagrams for explaining a pattern production/registration process, based on the flow chart indi- 
cated in Rg. 15: 

55 Fig. 1 7 Is a flow chart showing an embodiment of an image sticking synthesizing processing operation of the digital 
camera indicated in Fig. 1 ; 

Figs. 1 SA to 18C are diagrams for explaining a synthesized image producing process, based on the flow chart indi- 
cated in Fig. 17; 
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Fig. 19 is a flow chart showing an example o1 a cut-out image production/registration processing operation; 
Figs. 20A to 20E are diagrams for explaining a cutout image production/registration process, based on the flow 
chart indicated in Fig. 19; 

Fig. 21 is a flew chart showing an embodiment 0^ a guiding frame display processing o^ 
5 indicated in Fig. 1 , taking photographing for a certificate using a photographic f ranrie for certificate as an exarnple; 

Figs. 22A to 22E are diagrams fbr explaining a photographing for a certificate, based on the flow chart indicated in 

Fig. 21; 

Figs. 23A to 23D are diagrams for explaining an example of the mask pattern production by contour extraction; 
Figs. 24A and 24B are flow charts showing an example of the pattern producing operation, based on a distance 
10 designation; 

Fig. 25 is a flow chart showing an example of operation of the pattern producing means by a drawn pattern; and 
Figs. 26A to 26Q are diagrams for explaining formation of a mask pattern from a drawn pattern based on the flow 
chart shown in Rg. 25. 

15 ( EXAMPLES OF THE CIRCUIT CONSTRUCTION > 

Fig. 1 is a block diagram showing an example of construction of a digital camera, which is an embodiment of the 
imaging apparatus according to the present invention, in which the digital camera 1 00 is so constructed that a lens block 
41 is connected with a camera body 42 through a connecting section 43. 

20 The lens block 41 includes an imaging lens 1 . a CCD (Charge Coupled Device) 2, an amplifier 3, an A/D converting 
circuit 4 and a driving circuit 5 and it is 80 constructed that the imaging lens 1 fbcusee an optical image taken-in from 
an object to be imaged on the CCD 2. 

It is further so constructed that the CCD (imaging means) 2 converts light injected through the imaging lens (imag- 
ing means) 1 by the photoelectric effect and outputs a signal to the A/D converting circuit 4 through the amplifier 3 and 

25 the signal is oulputted to the camera body 42 through a terminal 31 in the connecting section 43 after having been A/D 
converted. 

The driving circuit 5 drives the CCD 2, when a control signal is inputted from the camera body 42 through a terminal 
32 in the connecting section 43. 

The camera body 42 includes a timing generator 12. a DRAM (Dynamic Random Access Memory) 13. a flash 
30 memory 14. a data compression/extension circuit 15, a signal generator 16, a VRAM (Video Random Access Memory) 
17, a D/A converting circuit 18, an amplifier 19, a liquid crystal display (LCD), a CPU (Central Processing Unit) 21, a 
key input section 24 and an I/O port 25 and it is so constructed that image data (digital data) coming from the A/D con- 
verting circuit 4 through the terminal 31 in the connecting section 43 are supplied to the timing generator. 

The timing generator 1 2 controls the driving circuit 5 through the terminal 32 to drive the CCD 2. The timing gener- 
35 ator 12 writes also the image data supplied from the A/D converting circuit 4 in the DRAM 13. The image data written 
in the DRAM 1 3 has color components of yellow (Ye), cyan (Cy) and green (Or). 

In the DRAM 13, apart from a region, where the image data are stored temporarily, tfiere is secured a work area 
for image processing (working region) at executing image processing treatment according to the present invention. 

The image data written in the DRAM 13 are read-out with a predetermined period, when image data of one frame 
40 are accumulated, to be supplied to the signal generator 16. 

Further the image data written in the DRAM 1 3 are compressed in the compression/extension circuit 15 and written 
in the flash memory 14 (taken image memorizing means) to be stored (record) there. 

In the flash memory 14, apart from a region, where image data of a taken image are stored, there are disposed a 
cut-out image memory region 14A (cut-out image memorizing mearYs), where a cut-out image is stored, and a mask 
46 pattern memory region 1 48 (mask pattern memorizing means). Furlher the clipped image memory regkxi 14A and the 
mask pattern memory region 14B may be disposed in another memory medium, e.g. an ROM or a memory card. 

The signal generator 16 takes-out the image data of one frame from the DRAM 13 with a predetermined period, 
which are sut>jected successively to while balance regulation and cok>r processing treatment to produce YUV data 
(luminance signal Y as well as chrominance signals R - Y and B - Y), which are stored in the VR/MV1 17. 
60 Further the signal generator 1 6 reads-out the YUV data stored in the VRAM 1 7 to output them to the D/A converting 
circuit 18. Then it outputs them to the lk|uid crystal display device 20 through the amplifier 19 to display the image after 
having D/A-converted them. Video signals outputted by the amplifier 19 can be outputted also to an external device not 
indicated in the figure 

Owing to the construction described above, the liquid crystal display (LCD) 20 can display the image of the object 
55 through the CCD 2 at imaging (through-image display) and display also a reproduced recorded image at image repro- 
duction. Since images of the object displayed at imaging are displayed as a moving image through the imaging lens 1, 
(that is, monitor images (hereinbelow called through-images) depending on movement of the object or the digital cam- 
era are displayed), the liqukJ crystal display 20 can be used as a finder. 
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Further, in Fig. 1 , a transparent position detecting sensor (not indicated in the figure) may be disposed on the 
screen of the liquid crystal display section 20. As the position detecting sensor e.g. light detecting elements or touch 
electrodes detecting specified light can be used. In case where light detecting elements are used as the position detect- 
ing sensor, a light detecting element which is at a pointed position is turned to an active state by the fact that a user 

5 points the position on the screen by means of an input pen emitting the specified light (e.g. infrared) as inputting means. 
The CPU 21 detects coordinates of light detecting elements turned to tiie active state and the user can know the posi- 
tion and the extent (form) of a pointed image. In case where tough electrodes are used, electrodes at a touched part 
are tumed-on. when the user touches the part on the screen with a touch pen or the extremity of a finger. The CPU 21 
detects coordinates of touch electrodes which are turned-on and the user can knew the position and the extent (form) 

10 of a pointed Image. 

The CPU 21 executes various sorts of processing according to programs stored in the ROM 22 or the flash memory 

14. 

The RAM 23 memorizes appropriately programs and data, etc. necessary 16r execution of the various sorts of 
processing by the CPU 21. 

IS The key input section 24 is pnovkied vMth various sorts of input keys or buttons, as lndk:ated in Fig. 2, and sends 
state signals of corresponding keys or buttons to the CPU 21 . Pointing devices such as an input pen. a touch pen, etc. 
may be added thereto apart from the key input section 24. 

The I/O port 25 executes interface processing for data in- and output tDetween the CPU 21 and external devices. 
Further distance measuring means (not indicated in the figure) capable of measuring the distance to the object to 

20 be imaged may be disposed in the lens block 41 or the camera body 42. Apart from that the distance measuring means 
can be mechanical means, for example, in case where the digital camera has an autotocusing mechanism, it can be 
constructed as means for obtaining tfie distance to the object to be imaged b obtained by an autofocusing operation, 
starting from the focal length f of the imaging lens 1 , a distance a between the imaging lens 1 and the CCD 2, and a 
displacement distance of the imaging lens 1 . The distance measuring means can be constructed also by using an inf ra- 

2S red emitting and receiving device. 

( EXAMPLE OF CONSTRUCTION OF THE KEY INPUT SECTION) 

Rg. 2 is a diagram showing a representative example of arrangement of keys or buttons In the key input section 24 
30 disposed in the digital camera, to which the present invention is applied. 

In Rg. 2, a frame button 61 is operated by the user, when a cut-out image Is read-out from the cut-out image mem- 
ory 14A; displacement keys 62 are operated, when a cursor displayed on the liquid crystal display 20 Is displaced up-, 
down-, left- or rightward; a release button 63 is operated, when a taken image is stored in the flash memory 14; and a 
reproduction button 64 is operated, when a reproduction mode Is set for the purpose of reproducing one of images 
35 stored in the flash memory 14 Cmduding taken Images, previously registered images, nnorKx;hrome images. Images 
registered in an IC card, eta). 

Pyrther a recording button 65 Is o perat ed, when a rec»rd mqde^^^ for the purpose of recording a taten image 
in the flash memory 14; a mask button 66 is operated, when a mask pattern (image) is reproduced; up-down keys 67 
are operated, when the cursor 20 displayed on the liquid crystal display 20 or in case where an instruction to start exe- 
40 cuton of a function prescribed in a program, etc. are carried out; and a direct key 68 is operated in case where an 
instruction to start drawing by using a cursor or a pointing device (e.g. an input pen. a touch pen, etc.) Is sent to the CPU 
21. 

< PROCESSING MODES) 

45 

Fig. 3 is a diagram for explaining an example of construction of the processing modes of the digital camera indi- 
cated in Fig. 1 . The processing modes of the digital camera can be classified roughly into a usual nrxxie consisting of a 
recording mode and a reproduction mode and a special imaging mode such as a dose imaging, etc. and switching-over 
of the usual mode and the special imaging mode is effected by switching-over of a processing mode switching-over 
60 switch. 

When the processing mode is set at the usual mode, it proceeds at first to the recording mode. The recording mode 
includes an imaging mode, by which imaging of an object and through-image display are effected, a through-Image syn- 
thesizing mode, and a record conservation mode, which a desired through-image, a cynlheslzed Image, etc. are 
recorded In a recording medium (the flash memory in the embodiment). 
55 When the reproduction button 64 is pushed under the usual mode, the process proceeds at first to the reproduction 
mode. The reproduction mode includes a reproduction/display mode, by which an image recorded in the flash memory 
14 is reproduced to be displayed, a recorded image synthesizing mode, and an output mode, by which image data 
(including also synthesized image data) can be outputted to an external device through the I/O port 25. 
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Mode judging means (constructed by a program in the embodiment) judges to which mode the process has pro- 
ceeded by operation o1 the processing mode switching-over switch, key operation or button operation in the key input 
section 24 by the user. The mode judging means examines state signals from the keys and the buttons sent from the 
key input section 24 to the CPU 21 and the process proceeds to a corresponding mode processing circuit or program. 

5 

(OPERATION OF THE DIGITAL CAMERA UNDER DIFFERENT IS/IODES> 

Now operation of the digital camera indicated in Fig. 1 under different modes will be explained. 

10 [OPERATION UNDER THE RECORDING MODE] 

When the processing mode is set at the recording mode, the process proceeds at first to the imaging mode. By the 
imaging mode, image data of one franfie taken-in periodically from the CCD 2 are displayed on the liquid crystal display 

20 as a through-image. 

When the mask button 66 or the frame button 61 is pushed under the imaging mode, the process proceeds to the 
through-image synthesizing nxxJe. by which it is possible to form a mask pattern starting from a desired image, to syn- 
thesize an image by combining an image and a desired mask pattern or a cut-out image to be displayed, to effect image 
synthesis, etc. 

Further, when the up-down key 67 is pushed* it is possible to display a guiding frame fbr imaging. On the other 
hand, when the release button is pushed, the mode proceeds to the record conservation mo6e, by which ft is possible 
to record a desired through-image, a synthesized image, etc. in the flash memory 14. 

(1) Imaging and display of a througl>image 

26 When the user pushes the recording button 65 under the recording mode, the CPU 21 sends an imaging instruction 
signal to the timing generator 12 to control is so as to effect taking-in of an image of an object. When the timing gener- 
ator 1 2 receives the imaging instruction signal from the CPU 21 . ft drives the driving circuit 5 through the terminal 32 so 
as to take-in a video signal converted by the CCD 2. Since an optical image of an object, to which the imaging lens 1 is 
directed by the user, is focused on the CCD 2. the video signal photoelectrically converted by the CCD 2 is inputted to 

30 the A/D converting circuit 4 through the amplifier 3 owing to the fact that the CCD 2 is driven by the driving circuit S 

The A/D converting circuit 4 A/D-converts the inputted video signal to have image data, which are supplied to the 
timing generator 12 through the terminal 31. 

The timing generator 12 stores the inputted image data (Ye, Cy, Gr) in the DRAM 13. The CPU 21 reads-out the 
image data, which have been once stored in the DRAM 13. to supply them to the signal generator 16. 

35 The signal generator 1 6 regulates whfte balance of these image data and at the same time it produces video data 
(YUV data) by encoding processing, which are temporarily stored in the VRAM 17. Further the signal generator 16 
reads-out the vi deo data dr awn in the VRAM 1 7 to supply ther ri to th e D/A coriverting circuit 18. 

Ttie D/A converting circuit 18 D/A-oonverts these video data and outputs them to the liquid crystal display 20 
through the annplrfier 19 to t>e displayed there. In this way the Image (through-image), which the user will monitor 

40 through the imaging lens 1 , is displayed on the liquid crystal display 20. It is possible for the user to verify the object to 
be imaged by observing the image displayed on this liquid crystal display 20. 

Further ft is possible to produce in a simple manner video data for the through-image displayed on the liquid crystal 
display 20. For example, since luminance data and chrominance data can be produced by using only the Ye connponent 
among image data stored in the DRAM 13, ft is possible to update rapidly an ever changing image (i-^- through-Image) 

46 coming through the imaging tens 1 by using these data to display ft on the liquid crystal display 20. 

(2) Synthesis of a through-image 

At imaging mode, when the user pushes the mask button 66 or the frame txjtton 61 , the process proceeds to the 
60 through-image synthesizing mode. 

In case where the mask button 66 is pushed, ft is possible to cut-out a desired part of a through-image as a mask 
pattern to register it in the mask pattern memory 1 4B or to combine a desired part of the through-image with a desired 
mask pattern to synthesize a desired cut-out image. On the other hand, when the frame button 61 is pushed, ft is pos- 
sible to stick a desired cut-out image recorded in the cut-out image (clipped image) memory 14A to the through-image 
65 to synthesize a new image. Further operation of the digftai camera 1 00 under the through-image synthesizing mode will 
be described later. (Figs. 4A-4C, Embodiments 1 to 3, 7 and 8 to 10) 
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(3) Display of a guiding frame 

When the up<Iown key 67 is pushed under the imaging mode, it is possible to display a guiding frame for imaging 
for a driving license, a photograph for passport, etc. Display operation of the guiding frame will be described later. 

5 

(Embodiment) 

(4) Record conservation of an image 

10 On the other hand, when recording is instructed by operating the release button 63 under the recording nxxle. the 
CPU 21 supplies innage data (including synthesized image data synthesized under the inrtage synthesizing mode) 
stored in the DRAM 13 to the signal generator 16 to have it produce (encode) video data more precisely by using all of 
the Ye component, the Cy component and Qr component to display them on the liquid crystal display 20 as a still image. 
Further the CPU 21 supplies these video data to the compression/extension drcuK 1 5 and compression processing 

IS is executed there e.g. by the JPEG (Joint Photographic Experts Group) method. The compressed video data are 
recorded (written) in the flash memory 14. 

Moreover information relating mask patterns, synthesized inwges and their bad^round images (recorded images 
in this case] (e.g. image numbers of the synthesized images and the background images, sort of the mask patterns and 
their image numt>ers, image numbers of cut-out images, etc.) may be written in a link table 50 disposed on the flash 

so memory 14 while making them correspond to each other. Further storage positions of different images and mask pat- 
terns may be written-in instead of the image numbers. 

[OPERATION UNDER THE REPRODUCTION MODE] 

26 When the reproduction txjtton 64 is pushed under the usual rrxxJe, the process proceeds to the reproduction/dis- 
play mode, a recorded image recorded in the flash memory 14 is read-out and reproduced to be displayed on the liquki 
crystal display 20. Every time the reproduction button is pushed, a succeeding image is read-out and reproduced to be 
displayed on the Ikiuid crystal display 20. In this way it is possit>le for the user to obtain a desired image. 

The apparatus may be so constructed that a fast-forwarding nxxJe may be added thereto, by whk:h, when the repro- 

30 duction button is continuously pushed, displayed images are fast forwarded and when a finger is detached from the 
reproduction button, an image is displayed after that point of time. 

When the mask txJtton 66 for the release button 63 is pushed under the reproduction/display mode, the process 
proceeds to the recorded image synthesizing mode, by which it is possible to superpose a desired mask pattern or a 
cut-out image on a recorded image or to stick a desired cut-out image to a recorded image. 

35 

(1) Reproduction and display of a recorded image 

When the user operates tiie reproduction button 64 in ttie key input section 24 to set ttie reproductionydisplay mode, 
the CPU 21 reads-out video data of one image written in the flash memory 14 to supply them to the compression/exten- 
40 sion circuit 15. 

The compression/extension circuit 1 5 subjects them to extension processing to supply them to the signal generator 

16. 

The signal generator 16 adds a synchronizing signal, etc. to the received video data and reads-out the video data 
recorded in the VRAM 17 after having stored them temporarily in the VRAM 17 to supply them to the D/A converting 
45 oYcuit 18. 

The D/A converting circuit 18 D/A-converts these vkJeo data and outputs them to the Ikiiuid crystal display 20 
through the amplifier 19 to be displayed there. In this way it is possible to have the liqukJ crystal display 20 display the 
image recorded in the flash memory 14. Since this displayed image differs from the through-image, as described pre- 
viously, and it is one taken-in after more precise processing, it can be displayed in a form clearer than the through- 
so image. 

(2) Synthesis of a recorded image 

When the mask button 66 or the release button 63 is pushed under the reproduction/display mo6e, the process pro- 
55 ceeds to the recorded image synthesizing mode. In case where the mask button 66 is pushed, it is possible to produce 
a desired mask pattern, starting from a recorded image, as a mask pattern, to superpose a desired pattern on the 
recorded image to be displayed, to stick a desired pattern to the desired recorded image to synthesize a new image. 
In case where there is disposed a link table 50 and informatk>n relating mask patterns, cut-out images, synthesized 
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images and their background images to each other is ther«)ut therein, this information is updated at record conserva- 
tion. Further operation under the recorded image synthesizing mode will be described later. (Figs. 4A to 4C, Embodi- 
ments 4 to 6. 9 and 10) 

5 ( IMAGE PFKDCESSiNG IWIEANS ) 

Image processing means 100 executes image processing treatmerrt under the usual mode. Although the image 
processing means 100 can be constructed by hardware ctrcuits, in the present embodiment the image processing 
means 100 is constructed by a program. Further certain modules among different modules of the image processing 
10 means 100 may be constructed by hardware circuits, while other modules are constructed by a program. 

The different nruxjules of the image processing means 100 constructed by a pros^ani are recorded in the ROM 22 
or the flash memory 14 and executions thereof are controlled by the CPU 21 under control of a control program to real- 
ize various sorts of image processing treatments in the present embodiment. 

Figs. 4A to 4C are block diagram shewing an example of construction of the image processing means 100. The 
IS image processing means 100 includes through-image synthesis processing means 110. recorded image synthesis 
processing means 120 and inner frame display processing means 130. 

[THROUGH-IMAGE SYNTHESIS PROCESSING MEANS] 

20 Through-image synthesis processing means 110 includes processing method judging means 111, which judges an 
image processing method, in case where the process is switched-over from the inf>aging mode to the through-image 
synthesizing mode under the recording mode; pattern productionAregistration means 122, which extracts a desired 
mask pattern from a through-image to register it in the mask pattern memory 1 4B; cut-out image production/registration 
means 1 1 3. which superposes a desired mask pattern on a through-image to display it and produces a desired clipped 

26 image, starting from the through-image superposed and displayed and the desired pattern, to register It in the cut-out 
Image memory 14 A; and through-image synthesizing means 114, which superposed a desired cut-out image on a 
through-image to be displayed and subjects the through-image and the cutK>ut image to sticking synthesizing process- 
ing. 

The processing method judging means 1 1 1 is started, when the process proceeds from the mode judging means 

30 to the through-image synthesis processing means 1 10 and judges a result of operation (state signal) of the user, which 
the CPU 21 has received from the key input section 24. in case where the result of operation of the user represents 
push-down of the mask button 66, it starts the pattern production/registration means 113 and for push -down of the 
release button It starts the through-Image production/registration means 114. 

As an option, in case where the link table 50 is disposed in the flash memory 14, processing information giving 

35 means 1 15 producing information registered in the link table 50 irx^luding e.g. image numbers indicating relations 
between background images and synthesized images, pointer information indicating relations t>etween bad^round 
images andj:ut-out innagi^ or nnask patternSj^eto. may be constructed so as to be incorporated in^ajatter stage of the 
through-Image synthesizing means. 

The pattern production/registration means 112 includes binary coded pattern production means 1 121 and lumi- 

40 nance component processing pattern productk>n means 1 1 22. 

The pattern production/registration means 1 12 is started by the processing method judging means 111. in case 
where the user pushes the mask button 66 at a desired angle while observing a through- image, when he finds an object 
desired as a mask pattern and images it under the imaging mode or when he draws a pattern to be registered as a 
mask pattern on a sheet of paper or a plate, which he wants to register as a mask pattern, arxl images it (e.g. Fig. 8A), 

45 takes-in the through-image (pattern carxikiate image) in the DRAM 13, and starts the binary coded pattern production 
means 1 121 so that the taken-in pattern candidate image binary coded to display the binary coded pattern (Fig. 8B) on 
the fiquld crystal display 20. In this way, when the up-down key 67 is p)ushed. luminance component processing pattern 
production means 1 122 is started, which takes-in the luminance component of the pattern candidate image in a prede- 
termined number of bits (6 bits in the emtxxilments) to modify the luminance component and displays an extracted pat* 

60 tern (Fig. 8C) on the liquki crystal display 20. Here, when the release button 63 is pushed, the extracted pattern is 
registered (recorded) in the mask pattern memory 14B. (EMBODIMENT 1) 

The pattern production/registration means 112 may be constructed so as to include pattern production means 
1 125. by which a pattern of an object image desired as a mask pattern is extracted by effecting a contour extracting 
processing for detecting the contour of the object to extract it. 

55 Further the pattern production/registration means 1 1 2 may be constructed so as to include pattern production 
means 1 1 25. so constructed that a pattern is extracted by the fact that the user moves the cursor on the screen of the 
Ik^ukJ crystal display section 20 by using the displacement keys 62 and traces the contour of an image of a desired 
ol:>ject as a mask pattern. Moreover the pattern production means 1 125 may be so constructed that the mask pattern 
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can be extracted by tracing the contour of the image of the object in the through-image, in case where, in Fig. 1 , as input 
means other than the keys and the buttons in the key input section 24 indicated in Fig. 2, the liquid crystal display sec- 
tion 20 is so constructed that a transparent position detecting sensor (not indicated in the figure) is disposed on the 
screen and the user can indicate a pattern thereon by means of a pointing device (not Indicated in the figure) such as 

5 a tough pen. etc. or a finger. (Embodiment 8) 

Mask patterns to be registered may be diagrams, designs and frames (e.g. a frame tor imaging having a size of a 
photograph for driving license, a frame for imaging having a size of a photograph fbr passport, a frame fbr imaging indi- 
cating a size of a desired image, etc.). 

Further they may be various general objects to be imaged such as matters, animals, plants, building, attached 

10 installations as signboards, landscapes, natural phenomena, etc.. if they can be imaged and extracted as nnask pat- 
terns. Moreover, for a plurality of imaged objects included in such a general image, it is possible also to produce a mask 
pattern at reproduction by extracting parts con^espondlng to a same distance. In this case, the pattern production/reg- 
istration means 1 12 Is so constructed that it comprises distance data storing means 1 126, which obtains distances from 
distance measuring means to objects to be imaged at a plurality of points in an image and stores them together with 

IS image data. (Embodiment 9) It is preferable to write measured distance data in the link table 50, which will be described 
later. 

The pattern production means 1 1 25 may t»e so constructed also that the user draws a trajectory by nfx)ving the cur- 
sor on the screen of the liquid crystal display section 20 by means of the displacement keys 62 and a figure thus drawn 
is cut-out as a mask pattern. In case where the liquid aystal display section 20 is so constructed that a transparent posi • 
20 tion detecting sensor (not indicated in the figure) is disposed on the screen and the user can Indicate a pattern thereon 
tjy means of a pointing device (not indicated in the figure) such as a tough pen, etc. or a finger, the pattern production 
means 1 125 may be so constructed that the user draws a figure while pointing on the screen of the liquid crystal display 
section 20 and the figure thus drawn is cut-out as a mask pattern. Further, in case where the position detecting sensor 
is constructed by a contact type detecting sensor such as touch electrodes, a figure, which the user has drawn by touch- 
as ing the screen with a finger, can be cut-out. (Embodiment 1 0) 

The cut-out innage production/registration means 113 cuts-out a desired part in a through-image by using a desired 
mask pattern registered in the mask memory 1 4B to register (record) rt in the cut-out image menrx>ry 1 4A. (Embodiment 
2) 

In this case, the user selects desired one anmng mask patterns displayed on tlie screen. In case where a desired 
30 part of a through-innage is cut-out by using a selected mask pattern, the mask pattern arxl the through-image are com- 
bined on the screen to be displayed on the liquid crystal display 20 in order to position the through- image (or the mask 
pattern) thereon. IHowever, since such an image display (hereinbelow called preview image display) takes much time, 
the cut-out image productton/registration means 1 1 3 can be so constructed that it comprises display coordinate deter- 
mining means, which effects a preview display at a high speed, in order to effect the preview display nx>re rapidly. 
35 (Embodiment 11) 

The through-image production means 114 combines a cut-out image registered in the cut-out image memory 1 4A 
with a desire d infiage among through-Image under the imaging mode and registers (records) an irnage thus syrrthesized 
in the flash memory 14. (Embodiment 3) In tNs case it is possible for the user to move the through-image by moving 
the camera so that the cut-out image Is superposed thereon at a desired angle. Further it is possible also not to move 
40 the camera, but to nx^ve the displayed cut-out image so that it is superposed on a desired part of the through- image. 

In this case, the user selects desired one among cut-out images displayed on the screen. Further, in case where 
the synthesis is effected by sticking the selected cut-out image to a desired part of the through-image, the cut-out image 
and the through-image are displayed on the screen to be positioned thereon. The through-Image synthesizing nr>eans 
114 can be so constructed that it comprises display coordinate determining means, whk:h effects a preview display at 
46 a high speed, in order to effect the preview display more rapidly, similarly to the cut-out image production/iregistratlon 
means 113. (Embodimerrt 1 1) 

The through-image synthesizing means 114 can be so constructed also that it conprises resolution determining 
means, which determines automatically resolution in case where a synthesized image is compressed to be registered 
in the flash memory 1 4 by using resolution of the cut-out image, resolution of the through-image, eto. (Embodiment 1 2) 

60 

[RECORD IMAGE SYrslTHE SIZING PROCESSING MEANS] 

The recorded image synthesizing processing means 120 includes the processing method judging means 121, 
which judges the image processing method in case where the nxxJe is switched-over from the reproduction/display 
55 mode to the recorded image synthesizing mo6e under the reproduction mode: the mask pattern production/registration 
means 122, which extracts/j^roduces a mask pattern from a recorded image to register rt in the mask pattern memory 
14B; the cut-out image production/registration means 123, which superposes a desired pattern on a recorded image to 
display it and cut-out a cut-out image from the recorded image thus superposed and displayed by using the desired pat- 
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tern to effect processing for producing a desired cut-out image; and the recorded image synthesizing means 124, in 
which the desired cut-out image is superposed on the recorded image to display them and the desired cut-out Image is 
stuck to the desired recorded image. 

As an option, in case where the link table 50 is disposed in the flash memory 14, processing information giving 
5 means 125 producing information registered in the link table 50 including e.g. image numbers indicating relations 
between background images and synthesized images, pointer information Indicating relations b^een background 
images and cut-out images or mask patterns, etc. may be constructed to as to be incorporated in a latter stage of the 
recorded image synthesizing means 124. 

The processing method judging means 121 is started, when the process proceeds from the mode judging means 
10 to the recorded image synthesizing processing means 1 20. and a judges a result of operation (state signal) of the user, 
which the CPU 21 has received from the key input section 24. In case where the result of operation of the user is push- 
down of the mask button 66. it starts the pattern production/registration means 122. On the other hand, for push-down 
of the frame button 61 it starts the cut-out image production/b'egistration means 123 and fbr push-clown of the release 
button it starts the recorded image synthesizing means 124. 
IS The pattern production/registration means 122 includes binary coded pattern production means 1221 and lumi- 
nance component processing pattern production means 1 222. 

Here, since the binary coded pattern production means 1221 and the luminance component processing pattern 
production means 1222 have same constructions as the binary coded pattern production means 1121 and the lumi- 
nance component processing pattern production means 1 122 descnbed previously, it is preferable that they are con- 
20 structed as common modules and that when the pattern production/registration means 112 of the pattern 
production/registration means 122 is started, the circuit is branched to these nrxxiules and as soon as the processing 
is terminated, they are returned to the pattern production/registratk)n means 1 12 or the pattern production registration 
means 122. 

The pattern production/registration means 122 is started by the processing method judgir^g means 121. in case 

23 where the user finds a image desired as a mask pattern among recorded images reproduced and displayed under the 
reproduction/display mode and pushes the mask button 66, writes the image in a work area disposed in the DRAM 13, 
and makes the circuit branch to the binary coded pattern production means 1221 . When the processing is terminated. 
It returns and an extracted binary coded pattern (Fig. 16C) is displayed on the llqukJ crystal display 20. In this case, 
when the up-down key is pushed, the circuit branches to the luminance component processing pattern production 

30 means 1222 and when the processing is terminated, it returns and an extracted pattern (Fig. 16D) is displayed on the 
liquid crystal display 20. Here, when the release button 63 is pushed, the extracted pattern is registered (recorded) in 
the mask pattern memory 14B. (Embodiment 4) 

The pattern production/registration means 122 may be constructed so as to include the pattern production means 
1 225. for extracting a pattern of an object image desired as a mask pattern by effecting a contour extracting processing, 

35 similarly to the pattern production/registration means 112 described previously. 

Further the pattern production means 1 225 may be so constructed that, when the user designates a distance at 
reproduction, on the basis of the stored distance data, parts of an Jmage. which are in accordance therewith, are 
extracted to produce a mask pattern. (Embodiment 9) 

Further, similarly to the pattern production/registration means 1 12 described previously, the pattern production/reg- 

40 istration means 122 may be constructed so as to include pattern production means 1225, in which the user moves the 
cursor on the screen of the liquid crystal display section 20 by using the displacement keys 62 to draw a trajectory and 
a figure thus drawn is extracted as a mask pattern. Similarly, in case where a transparent position detecting sensor is 
disposed on the screen of the liquid crystal display section 20 as input means and rt is so constructed that relevant 
points can be indicated by means of a pointing device, the pattern production means 1225 may be so constructed that 

4S the user draws a figure while pointing on the screen of the liquid crystal display section 20 and the drawn figure is cut- 
out as a mask pattern, ft is similar to that described previously at a point that, when the position detecting sensor is con- 
structed by contact type detecting elements, a figure drawn by the user while touching the screen with a finger can be 
cut-out. (Embodiment 10) 

Moreover, similarly to the case of the pattern production registration means 112, mask patterns to be registered 
60 may be diagrams, designs and frames (e.g. a frame for imaging having a size of a photograph for driving license, a 
frame for imaging having a size of a photograph for passport, a frame for imaging indicating a size of a desired image, 
etc.). matters, animals, natural phenomena, etc if they can be extracted as a mask pattern. 

The cut-out image production/registration means 123 cuts-out a desired part in a through-image by using a desired 
nrtask pattern registered in the mask pattern memory 14B to register (record) it in the cut-out image memory 14A. 
55 (Embodiment 5) 

In this case, the user specifies and selects desired one among mask patterns displayed on the screen. In case 
where a desired part of a recorded image is cut-out by using a selected mask pattern, the mask pattern and the 
recorded image are combined on the screen to be displayed on the liqukl crystal display 20 in order to position them 
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thereon. Similarly to the cut-out image production/registration means 113. the cut-out image production/registration 
means 1 23 can be so constructed that rt comprises display coordinate determining means, which effects a preview dis- 
play at a high speed. (Embodiment 1 1) 

The recorded image production means 124 combines a cut-out image registered in the cut-out image menrx>ry 14A 

5 with a desired part of a desired image anrtong reproduced recorded images and records an image thus synthesized in 
the flash memory 14. (Embodiments) 

It Is possible also for the user to move the displayed cut-out image so that the cut-out image Is superposed on the 
desired part of the reproduced image serving as a background. On the contrary, rt is possible also to move the repro- 
duced Image serving as a background so that the cut-out Image is superposed thereon at a desired angle. 

10 In this case, the user selects desired one among cut-out images displayed on the screen. Further, in case where 
the synthesis is effected by stiddng the selected cut-out image to a desired part of the recorded image, the cut-out 
Image and the recorded image are displayed on the screen to be positioned thereon. The recorded image synthesizing 
means 124 can be so constructed that it comprises display coordinate determining means, which effects a preview dis- 
play at a high speed, in order to effect the preview display more rapidly, similarly to the through-image synthesizing 

16 means 114, in order to increase the speed of such a preview display. (Embodiment 11) 

The recorded image synthesizing means 124 can be so constructed also that it comprises resolution determining 
means, which determines automatically resolution in case where a synthesized image is compressed to be registered 
In the flash memory 1 4 by using resolution of the cut-out image, resolution off the recorded image, etc. (Embodiment 1 2) 

20 [GUIDING FRAME DISPLAY PROCESSING MEANS] 

By using guiding frame display processing means 130 it is possible to obtain an image having a desired size by the 
fact that a guiding frame pattern such as a guiding frame for license photograph, etc.. registered in the mask pattern 
memory under the imaging mode, is displayed on the liquid crystal display 20 and that a photographer images/records 
26 while regulating the camera angle so that an image of an object is settled within the displayed gukJing frame. Further it 
is desirable to store size Information and images in the link table 50 at recording, making it correspond to images, in 
order that the image can be printed at a desired size at printing. 

< link table) 

30 

Figs. 5 A to 5C are diagrams for explanation indicating examples of construction of the link tatAe. The link table 50 
can store information relating synthesized images and their background images (recorded images in this case) to each 
other on the flash memory 14, i.e. image numbers of the synthesized images and their background image, sorts and 
image numbers of patterns, image nunrbers of cut-out images, etc. 

35 Fig. 5A indicates an example of the link table, in which synthesized images, background images, and cut-out 
Images are corresponded to each other, in case where the synthesized images are recorded in the flash memory 14 
after sticki ng sy nthesis, inrage ni^nbers of tt^ synthes ized i mages are stored in an image rTumber c olum n 51 1 ; image 
numbers of the background images are stored In another Image number column 512; and numbers of cut-out images 
stored in the cut-out image memory 14A are stored in a cut-out image nunrtoer column 513. 

40 Fig. 5B indicates an example of the link table, in which synthesized images are not recorded after sticking synthesis 
and informatksn making the synthesized images, background images, and cut-out images correspond to each other is 
registered, and in which numbers of synthesized cut-out images obtained by combining synthesized images with a 
bad^round image having an image number re^'stered in the image number column 512 are recorded in cut-out Image 
number columns 513-1 ~ 513-n (n^l). 

46 Fig. 5C indicates an example of the link table, in which informatkMi making cut-out images used at cut-out image 
production, background images (synthesized images may be used as backgrourxi images) and mask patterns corre- 
spond to each other is registered. Numbers of mask patterns used at producing cut-out images having cut-out Image 
numbers recoided in the cut-out image number column 51 3 among mask patterns recorded in the nnask pattern mem- 
ory 148 are recorded in a mask pattern number column 514. Position information (coordinates) of the background 

60 images, for which mask patterns are positioned on the background images at cut-out image productk>n. is recorded in 
a synthesis position column 51 5. 

In case where the pattern production/registration means 122 is constructed so as to include the image distance 
data storing means 1226. it is so constructed that there are disposed a plurality of distance columns for one inr^ge so 
that distance data to designated objects to be imaged can be recorded there. It is possible also that a print size column 

55 designating the size of printed images at image printing is disposed, in which a value of the print size or a code corre- 
sponding to the print size can be recorded. 
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(EMBODIMENTS) 

Figs. 6A to 6C are diagrams for explaining an example of synthesis (sticking synthesizing processing) of an image 
with a cut-out image under the through-image synthesizing nxxle or the recorded image synthesizing modB. 

5 It is supposed that an image indicated in Fig. 6A is a through-image displayed on the liquid crystal display 20 under 

the imaging nx>de or a recorded image reproduced under the reproduction mode. It is supposed further that a cut-out 
image as indicated in Rg. 6B is stored in the DRAM 13 or it is previously stored in the cut-out image memory 14A. In 
this case, in the sticking synthesizing processing the image indicated in Fig. 6A is combined with the cut-out image indi- 
cated in Fig. 6B to synthesize a new image and in this way a synthesized image indicated in Fig. 6C is produced. 

10 Hereinbelow embodiments under the through-image synthesizing mode and the recorded image synthesizing 
rvKXIe will be explained. 

[EMBODIMENT 1] Pattern production/registration processing (Mask pattern production/regi$tratk>n starting from a 
through-image) 

IS 

The present embodiment is one of the pattern production/registration means 112. which is an example, in which a 
desired mask pattern is extracted by background processing starting from a through-image under the through-image 
synthesizing mode to be registered in the mask pattern memory 14B. 

Fig. 7 is a flow chart indicating an embodiment of the pattern production/registration processing operation and Figs. 
20 8A to dC are diagrams for explaining a pattern production/iregistratton process on the basis of the flow chart indicated 
in Fig. 7. 

In Fig. 7, an image (in this embodiment a heart mark drawn on a sheet of paper or a plate) as irxik:ated in Fig. 8A. 
which the user wants as a mask pattern, is taken under the imaging mode. The object image (heart nnartO is taken tem- 
porarily in the DRAM 13 to display a through-image (T1). 
25 Then, when the user removes the camera while observing the throughnmage and operates the mask button 66 at 
a desired angle, the CPU 21 writes the image data stored temporarily in the DRAM 13 in a work area of the DRAM 13 
{T2). 

Here the CPU 21 examines whether the user has operated the up-down keys 67 or not In case where the up-down 
keys 67 have been operated, the process proceeds to T7 (T3). while in case where they have not been operated, the 
30 object image (image data) written in the work area is subjected to a background processing and further to a binary cod- 
ing processing after having been cut-out (T4). 

After the binary coding processing the signal generator 16 transforms these binary coded image data into video 
data, which are drawn in the VRAM 1 7, Thereafter the video data drawn in the VRAM 1 7 are read-out and outputted to 
the lk)uid crystal display 20 to be displayed there. In this way. a result (a through -image) obtained by processing a pat- 
35 terning candidate image, whk:h the user monitors through the imaging lens 1. into a binary coded pattern is displayed 
on the liquid crystal display 20. The user can verify the pattern (Fig. 88). which he wants, by observing this image dis- 
played on the lk)uid crystal display 20 (T5). 

Here the CPU 21 examines whether the user has operated the release button 63 or not. In case where the release 
button has been pushed-down, the process proceeds to T12. while it has not been pushed-down, the process returns 
40 toT3(T6). 

In case where the up-down key 67 has been operated in the step T3, it is examined whether a luminance compo- 
nent modification operation has been effected or not. Presence or at>serx:e of the luminance component modification 
operatton is judged by the fact that either the plus arxJ minus keys have been pushed-down at the same time or either 
one of them has been pushed-down (T7). In case where they have been pushed<iown at the same time, it is supposed 
46 that no modification operation has been effected and the process proceeds to T9 and on the other hand, in the case 
where either one of them has been pushed-down, it is sipposed that a modification operation has been effected and 
the process proceeds to T8 (T7). 

In case where the up-down key 67 (+) is operated in the step T3, the luminance value is shifted by a predetermined 
value at a time (1 by 1 in the present embodiment) towards higher values and on the other hand, in case where the up- 
60 down key 67 (-) is operated, the luminance value is shifted by a predetermined value at a time (1 by 1 in the present 
embodiment) towards lower values. Consequently it is possible to modify the luminance value of the pattern by operat- 
ing the up<jown key 67 (+) or the up-down key 67 (-) (T8). 

The luminance value of the patterning candkJate image written in the work area of the DRAM 13 is taken-in 8 bits 
to be patternized (T9). 

55 Every time either one of up-down keys 67 is operated, the signal generator 16 transforms luminance processed 
image data into video data to draw them in the VRAM 1 7. Thereafter the video data drawn in the VRAM 17 are read- 
out and outputted to the liquid crystal display 20 to be displayed. In this way a result (through-image) obtained by 
processing the patterning candidate image, which the user nrx>nrtors through the imaging lens 1 . into the luminance 
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processed pattern is displayed on the liquid crystal display 20. It is possible for the user to verify the pattern (Fig. 8C) 
by observing the Image displayed on the liquid crystal display 20 (T10). 

Here the CPU 21 examines whether the user has operated the release button 63 or not In case where the release 
button 63 has been operated, the process proceeds to T12 and on the other hand, in case where the release button 63 
5 has not been operated, the process returns to T7 (T1 1). 

In case where the release button 63 has been pushed-down in the steps T6 or T1 1 . i.e. in case where the desired 
mask pattern (Fig. 8B or 8C) has been ok>tained, the CPU 2 1 cuts-out the pattern displayed at that time as a mask pat- 
tern to register it in the mask pattern memory 14B (T12). 

The image of the luminance component processed pattern obtained in the step T10 is a pattern giving such a feel- 
10 ing that neighborhoods of the bourKiary of the pattern are blurred, as indicated in Fig. 8C. In this way it is possible to 
obtain a cut-out image giving a blurred feeling as indicated In Fig. 11 D stated later (cut-out image). 

According to the present enrt)odiment, since a drawn figure can be imaged and extracted/iregistered as a mask pat- 
tern, it is possible to register the mask pattern in an extremely simple manner. In addition, since preferred patterns can 
be designed and registered, e.g. trial productk>n of trade marks, corporation marks, etc. can be effected easily, owing 
IS to a synthesizing processing, etc. stated later, on the basis of a designed pattern. 

Although in the above explanation an example, in which a drawn figure is imaged and extracted/registered as a 
mask pattern, has been described, the present invention is not restricted thereto, but it is possible also to image an 
object, which the user wants to patternize, among landscapes of towns or streets, posters, signboards, magazines, etc.. 
to patternize it. 
20 As such examples, following can be cited: 

(A) As described in said Embodiment 1 . an example, in which the cursor is displaced on the screen: the contour of 

an image desired as a mask pattern is traced: and a pattern is extracted: 

(B) an example, in which a transparent position detecting sensor is disposed on the screen; and the contour of a 
26 through-image is traced to be extracted as a pattern by means of a pointing device, etc. such as s touch pens, etc. 

(Emtxxiiment 8): 

(C) an example, in which a figure, which the user has drawn by moving the cursor on the screen, is inputted to be 
extracted (Embodiment 9); and 

(D) an example, in which a transparent position detecting sensor is disposed on ttie screen; and a figure is drawn 
30 on the screen by means of a pointing device such as a touch pen. an input pen, etc. to be extracted as a mask pat- 
tern. 

[EMBODIMENT 2] Cut out image production/registration processing (cut-out image productiorVregistration processing 
starting from a through-image and a mask pattern) 

35 

The present embodiment is one of the image production/registration means 1 13. in which a through-image and reg- 
istered mask pattern are suitably combined to obtain a synthesized image, which will be a cut-out image, which is cut- 
out to k>e registered. 

Fig. 9 is a flow chart showing an embodiment of the cut-out image production/registration processing operation and 
40 Figs. 10A-10D and 11 A to 11 D are diagrams for explaining the cut-out image production/registration process, on the 
t>asis of the flow chart indicated in Fig. 9. Hereinbelow explanation will be made, taking a case where the user uses an 
object image as incScated in Fig. 1 0A as a cut-out image as an example. 

In Fig. 9, an object as ir>dicated in Fig. 10 A, which the user desires as a cut-out image, is imaged and an object 
image thus obtained is taken temporarily in the DRAM 13 to be displayed as a through-image (U1). 
46 Here the CPU 21 examines whether the mask button 66 has been operated or not (S2). 

When the user pushes the mask button 66 the CPU 21 reads-out a nr^ask pattern stored in the mask pattern mem- 
ory 14B: draws it in the VRAM 17 through the signal generator 16; reads-out the mask patten drawn in the VRAM 17; 
and displays it on the liquid crystal display 20 through the D/A converting circuit 18 and the anplifier 19. In this way. e g. 
a mask pattern as indicated in Fig. 108 is superposed on the through-image as indk:ated in Fig. 10A and displayed on 
60 the liquki crystal display 20 as indicated in Rg. IOC. In this case the mask pattern is positioned at the center of the 
screen (U3). 

Here, when the mask button 66 is pushed again, a succeeding mask pattern is read-out and superposed on tiie 
through-image to be displayed and the user judges whether the mask pattern displayed on the liquki crystal display 20 
is desired one or not. In case where it is not the desired pattern, the user continues to push the mask txitton 66. until 
55 the desired mask pattern appears. When it appears, it is combined with the through-image, superposed thereon, to syn- 
thesize a new image, which is displayed (U4). 

In the step U4, when tiie mask pattern is displayed, in case where the user judges that it is necessary to correct 
the position of the through-image, while observing the through-image, on which the mask pattern is superposed in the 
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step U4 (e.g. because a desired part of the through- image is not superposed on the nnask pattern at the cerrter of the 
screen), the user moves the camera and regulates the angle so that the desired part of the through-image is super- 
posed on the mask pattern at the center of the screen. In case where it is not necessary to correct the position of the 
through-image, the user pushes the release button (115). 
5 The CPU 21 examines whether the user has pushed the release button 63 or not (U6). In case where the release 

button 63 has been pushed, the part of the image (Fig. lOD) displayed on the liquid crystal display 20 is cut-out to be 
registered in the cut-out image memory 14A (U7). 

Although, in the step U5 in Fig. 9, the user has nfx>ved the camera to conrect the position of the through-image so 
that the desired part of the through-image is superposed on the mask pattern at the center of the screen, the step U5 
10 may be so constructed that the user operates the displacement keys 62 (or input means such as a pointing device, eta) 
to displace the mask pattern. 

When the cut-out image, which the user himself has made, is registered in the cut-out image memory 14A in the 
manner as described above, it is possible to eHect a synthesizing processing as described later (Embodiments 3 and 5). 

Fig. 10B is a binary coded pattern and in this case, the boundary of the cut-out image is clear, as indicated in Fig. 
IS 10D. On the contrary, if a cut-out image as indicated in Fig. 1 1 D is produced by using a luminance processed pattern 
as indicated in Fig. 11 B in the above embodiment, it is possible to obtain a cut-out image having a blurred boundary. 

Although, in the step U7 in Fig. 9, a part of image displayed on the liquid crystal display 20 is cut-out to be regis- 
tered, the apparatus may be so constructed that it is not registered at inrtaging, but it is registered at the cut-out image 
production/registration processing. 

20 

[EMBODIMENT 3] Sticking synthesizing processing (synthesis by combining a through-image with a cut-out image) 

The present embodiment is one of the synthesizing means 1 14, which is an example in which a through-image and 
a cut-out image registered in the cut-out image memory 14A are combined to synthesize a new innage under the 
2S through-image synthesizing mode. 

Figs. 12A and 12B are flow charts showing embodiments of the sticking synthesizing processing operation by 
means of the digital camera indicated in Fig. 1 and Figs. 13Ato 13D, 14Aand 14Barediagranr«fbr explaining thestick- 
ing synthesizing process, on the basis of the f k)w charts indicated in Rgs. 12A and 12B. 

In Fig. 12A, the user images a desired object under the imaging mode and an object image is taken in the DRAM 
30 1 3 to display a through-Image (VI ). 

Next, when the user pushes the frame button 61 (V2), the CPU 21 reads-out a first cut-out image among cut-out 
images recorded in the cut-out image memory 14A to supply it to the signal generator 16, which image is drawn In the 
VRAM 17. Since the through-image taken-in In the step V1 Is already drawn in the VRAM 17, as the result an image 
synthesized by combining the through-image with the reproduced cut-out image is drawn. This synthesized image is 
35 read-out by the signal generator 16 and outputted to the liquid crystal display 20 through the amplifier 19 after having 
been A/A-converted by the D/A converting circuit 18 so that it is displayed at a predetermined position (at the center in 
the present embo dimen t) of the liqu«l crystal display 20. In this wa y. e.o. a synthesized image as iridicated In Fig^13A 
is obtained. In Rg. 13A, 21 represents a through-image, while 22 Indicates a cut-out innage (V3) and 23 indicates a 
background image. 

40 The user judges whether the displayed cut-out image 22 is a desired cut-out image or not. while observing the syn- 
thesized image 20, as indicated in Fig. 13A. displayed on the Ikiuid crystal display 20. Now. in case where the displayed 
cut-out image 22 is not the desired cut-out image, the process returns to the step V2. The user continues to push the 
frame button 61 , until a desired synthesized image with the cut-out image is displayed. In this way the CPU 21 reads- 
out a cut-out succeeding Image recorded in the cut-out inr^ge memory 14A to display the synthesized image at the pre- 

46 determined position of the liquid crystal display 20 in the step V3 in a similar manner to the above described case. In 
case where the synthesized innage with the desired cut-out lavage is displayed, the process proceeds to V5 (V4). 

In case where the user judges that it is necessary to correct the position of the through-image 21 while observing 
the image displayed in the step V4 (e.g. in case where arrangement of the stuck image 22 is not preferable from the 
point of view of composition), the user moves the camera to regulate the angle so that the desired part of the through 

60 image 21 is superposed on the cut-out image 22 at the center of the screen (Fig. 13B}. On the other hand, in case 
where it is not necessary to correct the position of the through-image 21 as shown in Fig. 13B, the user pushes the 
release button (V5). 

In case where the desired synthesized image is obtained in the step V5, the user deckles whether it should be 
recorded in the flash memory 1 4 or not (V6). In case where he decides as a result off this selection that the synthesized 
55 image should be recorded in the flash memory 1 4, when the user operates further the release button 63 (V7), the CPU 
21 reads-out the synthesized Image data drawn at that time in the VRAM 1 7 and has the flash memory store them as 
one of recorded images (V8). In the case where it is decided as the resuH of the selection in the step V6 that the syn- 
thesized image need not be recorded in the flash memory 14, the synthesized image is only displayed on the Ikiuki 
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crystal display 20. 

Although, in the step V5 in Fig. 12A, the user has moved the camera to correct the position of the througivimage 
21 so that the desired part of the through-image is superposed on the stuck image 22, the step V5 may be so con- 
structed that the user operates the cursor displacement keys 62 (or Input means such as a pointing devk:e. etc.) so as 
5 to be able to displace the cut-out image. 

In this case, the apparatus can be so constructed that, in the case where it Is necessary to correct the position of 
the cut-out image in the step V5 as indicated in Fig. 12B, when the user operates a displacement key corresponding to 
a predetermined direction among the displacement keys 62 (VS*), the CPU 21 moves the cut-out image in the direction 
corresponding to the operated displacement key 62 to be displayed and the process returns to V5 (V5"), and that, in the 
10 case where it is not necessary to correct the position of the cut-out image as the result of the selection in the step V5 
(including cases wtiere it is OK as the result of corrections In the steps V5' and VS**) the process proceeds to V6. 

Figs. 13C and 13D show examples, in which the cut-out image is displaced. Since the cut-out Image 22 at the 
center is superposed on the background so that the background cannot be seen in Fig. 13A, the cut-out image 22 is 
displaced left-upward as indicated in Fig. 13B by operating displacement keys 62. 
IS Further, although the present embodiment is so constructed that the synthesized image data are recorded in the 
flash memory 14 as one of recorded images as described in the step V8. it is not restricted thereon, but it can be so 
constructed that the link table 50 descrbed previously is disposed in the flash memory 14 and at recording the synthe- 
sized image Information is registered in the link table 50, attaching another meaning thereto than the recorded image. 
It may be so constructed also that, instead of recording the synthesized image, only such information is registered in 
20 the link table 50 and the synthesized Image is reproduced on the basis of the information registered In the link table 50 
at displaying the synthesized image or outputting it to an external device. 

The cut-out image 22 used in Figs. 1 3A to 1 3D is a synthesized image with a binary coded pattern and in this case 
the boundary of the cut-out image 22 is clear. On the contrary, when a cut-out image 22' based on a synthesized image 
with a luminance processed pattern as indicated in Fig. 14 is used, it is possible to obtain a synthesized image having 
26 a blurred boundary contacted with the cut-out image. 

[EMBODIMENT 4] Pattern production/registration processing (production/registration of a mask pattern starting from a 
recorded Image) 

30 Although in Embodiment 1 descrik)ed previously a desired mask pattern is extracted from a through-innage to be 
registered, it is possible also to extract a mask pattern from a recorded image already registered. 

The present embodiment is one of the pattern production/registration means 122, in which a desired part of a 
reproduced desired recorded image is extracted as a mask pattern under the recorded image synthesizing mode to be 
registered in the mask pattern memory 148. 

55 Fig. 1 5 is a flow chart showing an emtxxiiment of the pattern production/registration processing operation and Rgs. 
16A to 16D are diagrams fbr explaining the pattern production/registration process on the basis of the flow chart indi- 
cated in Fig. 15^ _ 

In fig. 15, the user operates the reproduction button 64 to set the reproduction mode and reproduces a first 
recorded image to display it. 

40 The user can reproduce an image including an object, from which he wants to make a mask pattern, to display it 
on the Wquki crystal display 20. For example, a reproduced image (in thiis example an image of a picture, in which a pat- 
tern is drawn) as indicated in Fig. 16A is displayed on the liquid crystal display 20. Further, by a predetermined opera- 
tion the CPU 21 displays a frame type cursor 1 71 at a predetermined position on the screen (W1). 

Then, when the user posittons the cursor 1 71 so as to 6nck)se a desired image 1 72 therewith while observing the 

45 reproduced image and operates the mask t>utton 66, the CPU 21 reads-out data corresponding to the part enclosed by 
the frame of the cursor 171 from the vkJeo data (compressed data) recorded In the flash memory 14 to supply them to 
the data compression/extension circuit 15. Then it writes the video data subjected to extending processing (Fig. 16B) 
in a work area of the DRAM 13 (W2). In case where a desired image is greater than the cursor 1 71 in the step W1. the 
user pushes the up-down key 67 {^) to enlarge the cursor 1 71 so that the desired image can be enclosed. On the con- 

60 trary. in case where the desired image is too small with respect to the cursor 171, the user pushes the up-down key (-) 
to reduce the cursor 1 71 so that the size of the frame can be regulated. 

Here the CPU 21 examines whether the user has operated the up-down keys 67 or not (W3). In case where the up- 
down keys 67 has been operated, the process proceeds to W7 and in case where it has not been operated, the object 
image (ifnage data) written in the work area is cut-out and thereafter subjected to binary coding processing (W4). 

65 After the binary coding processing, the signal generator 16 transforms these binary coded image data into video 
data to be drawn in the VRAM 17. Thereafter the video data drawn in the VRAM 1 7 are read-out and outputted to the 
liquid crystal display 20 to be displayed there. In this way a result obtained by processing the reproduced patterning 
candidate image into a binary coded pattern is displayed on the lk)uid crystal display 20. It is possible for the user to 
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verify the pattern (Fig. 1 6C) by observing the image displayed on the liquid crystal display 20 {W5). 

Here the CPU 21 exannlnes whether the user has operated the release button 63 or not. In case where the release 
button 63 has been operated, the process proceeds to W12, while in case where it has not been operated, the process 
returns to W3(W6). 

5 In case where the up-down key 67 has been operated in the step W3, it is examined whether the luminance com- 

ponent modifying processing should be effected or not. and in case where it need not, the process proceeds to W9, 
while in case where It should be, the process proceeds to W8 (W7). 

in case where the luminance component modifying processing is effected in the step W7, when the up-down key 
67 (+) is pushed, the brightness is shifted by a predetermined value at a time (1 by 1 in the present embodiment) 
10 towards higher values, while when the up-down key 67 (-) is pushed, the brightness is shifted by a predetermined value 
at a time (1 by 1 in the present embodiment) towards lower values. Consequently the brightness can be varied by oper- 
ating the up-down key 67 (+) or the up-down key 67 (-) (W8). 

Tlie luminance component of the patterning candidate innage written in the work area in the DRAM 13 is takervin 
in 8 bits and processed to be patter nized {W9). 
IS Every time the up-down keys 67 is operated once, the signal generator 1 6 transforms these luminance component 
processing image data into video data to be drawn in the VRAM 1 7. Thereafter the video data drawn in the VRAM 17 
are read-out and oulputted to the liquid crystal display 20 to be displayed there. In this way a result obtained by process- 
ing the desired patterning candidate image, which the user has read-out from the flash memory 14, is displayed on the 
liquid crystal display 20. It is possible for the user to verify the pattern (Rg. 1 6D) by observing the image displayed on 
20 the liquid crystal display 20 (W10). 

Here the CPU 21 examines whether the user has operated the release button 63. In case where the release button 
63 has been operated, the process proceeds to W12 and on the contrary, in case where it has not been operated, the 
process returns to W7 (W1 1). 

In case where the release button 63 is pushed in the step W6 or W1 1 , i.e. in case where the desired rnask pattern 
2s (Fig. 16C or 16D) is obtained, the CPU 21 registers the displayed pattern image in the mask pattern memory 14B as a 
mask pattern {W12). 

According to the present embodiment, the luminance component processed pattern image obtained in the step 
W10 is one having a feeling that neighbortxxxis of the boundary are blurred as indicated in Fig. 16D and therefore it is 
possible to obtain a cut-out image having a blurred feeling as indicated in Figs. 14A and 14B (cut-out inrage). 
30 Further, according to the present embodiment, since it is possible to reproduce a recorded image and to extract/register 
it as a mask pattern, it is possible to take-out an image taken and recorded/conserved at a desired point of time and to 
fabricate a mask pattern in a simple operation. 

Although, in the present emtx>diment, a frame type cursor 1 71 as indicated in Fig. 16A is displayed in the step W1 
and the part enclosed by the cursor 171 in the reproduced image is extracted in the step W2, the present invention is 
35 not restricted thereto. Following examples can be cited as substituting embodiments; 

_ C^)^" ^ITPJ^- ^^ff^AP^Ir^ ^l^*!^!^ dispj^ed in the step vy^1 and tiie user moves the point curso r by operating 
the displacement l^ys 62 so as to trace the contour of the desired part of the reproduced Image to extract a closed 
domain, whose pattern is traced; 
40 (F) an example, in which a transparent position detecting sensor is disposed on the screen; the contour of the 
desired part of a reproduced image is traced by means of a pointing device, etc. such as a tough pen. etc.; and a 
closed domain, whose pattern is traced, is extracted (Embodiment 6); 

(Q) an example, in which distance data of a plurality of objects included in an image are recorded together with 
image data; the image is reproduced; and certain parts of a reproduced image are specified by designating a dis- 
46 tance to be extracted (Embodiment 9) ; and 

(H) further the pattem may be extracted by effecting the contour extracting processing by the fact that the user dis- 
places the cursor on the screen by using the displacement keys to trace the contour of tiie object to be imaged. 

[EMBODIMENT 5] Recorded image synthesizing processing (combining a recorded image and a cut-out image to syn- 
60 thesize a new image) 

The present embodiment is an example of the recorded image synthiesis by the recorded image synthesizing 
means 124. in which a recorded image recorded in the flash memory 14 is combined with a cut-out image with a frame 
registered previously in the cut-out image memory 14A to synthesize a new image under the recorded image synthe- 
55 sizing mode. 

Fig. 1 7 is a flow chart showing an embodiment of the recorded image synthesizing processing operation according 
to the present invention and Figs. 6A to 8C are diagrams for explaining the synthesized image producing process on 
the basis of the flow chart indicated in Fig. 1 7. 
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In Fig. 17. the user reads-out a recorded image used for a synthesis from the flash memory 14 and has the liquid 
crystal display 20 display it. This operation is effected by the fact that the reproduction mode is set by operating the 
reproduction button 64. that a first recorded image is reproduced to be displayed, and that a desired image is repro- 
duced/displayed by operating the up-down keys 67 (reproduction/display mode). In this way, e.g. a reproduced image 
5 as indicated in Fig. 1 8A is displayed on the liquid crystal display (81). 

Then, when the user operates the frame button 61 (S2), the CPU 21 reads-out a predetermined cut-out image 
among cut-out images recorded in the Image memory 14A to supply them to the signal generator 16, which is drawn in 
the VI^AM 1 7. Since the recorded image reproduced in the step SI is already drawn in the VRAM 1 7. as the result a 
synthesized image obtained by combining the reproduced image and the reproduced cut-out image is drawn in the 
10 VRAM 17. The synthesized image is read-out by the signal generator 16 and after having been D/A-converted by the 
D/a^nverting circuit 18 it is outputted to the liquid crystal display 20 through the amplifier 19 to be displayed. In this 
w^ e.g. a synthesized image as Indicated in Fig. 18B can be obtained (S3). 

The user decides whether the displayed image is the desired cut-out image or not while ot)serving the synthesized 
image displayed on the liquid crystal display 20. Now, in case where the displayed image is not the desired cut-out 
IS Image, the process proceeds to the step S2 and the frame button 61 is operated again. In this way the CPU 21 reads- 
out a succeeding cut-out image recorded in the cut-out image memory 14A and displays the synthesized image on the 
liquid crystal display 20 in the step S3 in a sirr^lar way to the case desaibed previously. The user continues to push the 
frame button 61 , until the desired synthesized image with the cut-out image is displayed. When the desired synthesized 
image with the cut-out image is displayed, the process proceeds to S5 (84). 
20 The user decodes whether it is necessary to correct the position of the image within the frame or not while observ- 
ing the synthesized image. As a result of this selection, in caee where it is necessary to correct the position of the image 
within the frame, the process proceeds to 88 (85). 

In the step 85. in case where it is necessary to correct the position of the image within the frame, when the user 
operates the displacement key corresponding to a predetermined direction among the displacement keys 62 (S6). the 
25 CPU 21 displaces/displays the reproduced image to the directton conresporKling to the operated displacennent key and 
the process returns to 85 (87). 

For example, when in a state where the synthesized image indicated in Fig. 18B is displayed, leftward and down- 
ward displacements are instructed by operating the corresponding displacement keys 62, the image displayed, super- 
posed on the cut-out image, is moved left- and downward as indicated in Fig. 1 8C. As described above, it is possible to 
30 move the background Image (reproduced recorded image) within the frame so as to position it at a desired position. 

In addition, rt is possible also not to move the background image but to move the cut-out image. 

In case where it is not necessary to correct the image within the frame as a result of the selection in the step 85 
(including cases where it is OK as the result of the position correction (86 and 87), the user decides whether the syn- 
thesized image displayed on the liqukJ crystal display 20 at that time should be recorded in the flash memory 14 or rx>t 
36 (S8). As the result of this selection, in case where tiie synthesized image shouki be recorded in the flash menrK>ry 14, 
the process proceeds to S9 and in case where it need not. the process is terminated. In this case the synthesized Image 
is only displayed on the \k\uidi crystal display 20. 

As the result of this selection, In case where the synthesized image shouM be recorded in the flash memory 1 4, 
when the user operates further the release button 63 (89), the CPU 21 reads-out the synthesized image data drawn in 
40 the VRAM 17 at that time and has the flash memory 14 store them as one of recorded images (810). 

Although the present embodiment is so constructed that tiie synthesized image data are recorded as one of 
recorded images as described in the step 810. the present invention is not resta^icted thereto, it may be so constructed 
that the link table 50 (Figs. 5 A to 5C) is disposed in the flash memory 14 and that attaching another meaning thereto 
than the recorded image, such information is registered in the link table 50. Furtiier it may be also so constructed tiiat 
46 instead of recording the synUiesized image only such information is registered in the link table 50 and that the synthe- 
sized image is reproduced on the basis of the information registered in the link table 50 at display of the synthesized 
image or output thereof at an external device. 

[EMBODIMENT 6] Cut-out image production/registration processing (production/registration of a cut-out image, start* 
60 ing from a recorded image and a mask pattern) 

Although the synthesizing processing described above is so constructed that a cut-out image with a frame previ- 
ously recorded in the cut-out image memory 14A is read-out appropriately and a desired image is selected among a 
plurality of cut<»ut images, it is possible also to produce a cut-out image by using a recorded image stored already in 
55 the flash memory 14. 

The present embodiment is one of the cut-out image production/registration processing by the cut-out image pro- 
duction/registration means 1 23. which combines appropriately a recorded image with a mask pattern prevtously regis- 
tered to synthesize a new image and cuts-out an image to register it. 
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Fig. 19 is a flow chart showing an embodiment oi the cut-out image production/registration processing operation 
and Figs. 20 A to 20 E are diagrams tor explaining the cut-out image production/registration process on the basis of the 
flow chart indicated In Fig. 19. 

In Fig. 19. the user reproduces a first image to display it by operating the reproduction button 64 to set the repro- 
5 duction mode and has the liquid crystal display 20 reproduce and display the image, from which the wants to make a 
cut-out image, by operating the up-down keys 67. In this way e.g. a recorded image as indicated in Fig. 20A is displayed 
on the liquid crystal display 20 (S21). 

Then, when the user operates the mask button 66. the CPU 21 reads-out mask patterns stored in the mask pattern 
memory 14B; draws them in the VRAM 17 through the signal generator 16 (S22); and reads-out the mask patterns 
10 drawn in the VRAM 17. which are displayed on the liquid crystal display 20 through the D/A converting circuit 18 and 
the amplifier 1 9. In this way e.g. mask patterns as indicated in Fig. 20B are displayed on the liquid crystal display 20. At 
this time the CPU 21 displays also the cursor, superposed on the mask patterns (823). 

Here the user selects one of the mask patterns displayed on the liqukl crystal display 20. That is. a cursor as indi- 
cated in Fig. 20 B (a frame type cursor in the example indicated in the figure) is positioned on a desired mask pattern by 
IS operating displacement keys 62. In order to fix tiie selection, the mask button is operated again (S24). 

The CPU 21 sends only the mask pattern selected in the step S24 to the VRAM 17. which is superposed on tiie 
reproduced image selected in the step S21 to be drawn. In this way e.g. the reproduced image irxitcated in Fig. 20A and 
the selected mask pattern indicated in Fig. 20B are superposed on each other to be displayed on the liquid crystal dis- 
play 20 as indicated in Fig. 20C (S25). 
20 Next, in case where the user judges tiiat it is necessary to correct tiie position of the mask pattern while observing 
the displayed image, in which the two are superposed on each other in the step S25, the process proceeds to S27 and 
otiierwise it proceeds to S29 (S26). 

In case where the user judges that it is necessary to correct the position of the mask pattern in the step S26, he 
operates the displacement keys 62 (327) to move the mask pattern. For example, if the upward key is operated anrx>ng 
26 the displacement keys 62 in the state indicated in Fig. 20C, tiie mask pattern moves upward, as indicated in Fig. 20D 
(S28). 

In case where the synthesized image is obtained by combining the desired mask pattern and tiie recorded image 
in the step S26 or S28. the user operates the release button 63 in order to register the synthesized image as a cut-out 
image in the flash memory (S29). 

30 At this t me the CPU 21 cuts-out the synthesized image (image indicated in Fig. 20D) displayed on the liquid crystal 
display 20 to record it as a cut-out image (pattern frame + image) in the cut-out image memory 14A (S30). 

When the cut-out image made by the user himself is registered in the cut-out image memory 1 4 A in the manner as 
described above, it is possible to effect ttie cut-out image production processing by an operation similar to that 
explained, referring to the flow chart indicated in Fig. 17 (refer to Embodiment 5). That is. when the frame button 61 is 

35 operated a predetermined number of times in the step S2 in Fig. 1 7. the image indicated in Fig. 20D is read-out from 
the cut-out image memory 14A to be reproduced. Then it is reproduced on the liquid crystal display 20 in the step S3. 
Conseq uentl y, Jf ttie recorded image combined with the cut-out image to produce the syrrttiesized Jniage is prewusly 
reproduced and displayed, it is possible to synthesize an image t>y combining ttie cut-out image made by htmseif with 
another recorded image, as indicated in Fig. 20E. Further it is possible to record also the image synthesized in this way 

40 in the flash memory 1 4 by operating the release button as stated in S9. 

As described above, in Embodiments 5 and 6. it is possible to synthesize a new image by combining an image 
recorded in the flash memory 14 with a cut-out image registered in the cut-out image memory 14A and ffurttier a cut- 
out image with a frame can be newly produced by combining appropriately an image and a mask pattern already 
recorded. In tiiis way a recorded image can be easily processed (combined to synthesize). 

46 

[EMBODIMENT 7] 

Fig. 21 is a flow chart showing an embodiment of the gukling frame display processing of the digital camera indi- 
cated in Fig. 1 , in which imaging of a certificate photograph using a frame for certificate photograph is taken as an 
60 example, and Figs. 22A to 22E are diagrams of the certificate photograph imaging processing on the basis of the fksw 
chart indicated in Fig. 21. 

In Fig. 21 , when the user operates the up-down key 67. tiie CPU 21 reads-out a mask pattern stored in the mask 
pattern memory 1 4B: draws it in the VRAM 1 7 through the signal generator 16: reads-out a guiding frame pattern drawn 
in the VRAM 1 7; and displays it on the ik:iuid crystal display 20 through tiie D/A converting circuit 18 and the amplifier 
55 1 9. In this way e.g. a guiding frame 81 as indicated in Fig. 22A (in this emtx>diment. a guiding frame for photograph for 
driver license) is displayed on the liqukj aystal display 20. In this case, the guidirtg frame 81 is positioned at the center 
of the screen (XI). 

Here, when the ip-down keys 67 are operated further, a succeeding guiding frame is read-out to be displayed. The 
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user judges whether the guiding frame displayed on the liquid crystal display 20 Is desired one or not. In the case where 
It is not the desired guiding frame, the up-dcwn keys 67 are operated, until the desired guiding frame appears, in order 
to display the guiding frame (X2). 

In case where the desired frame is displayed in the step X2, the user examines whether a through-image B2' (Rgs. 
5 22C and 22D) o1 an object to be imaged 82 (Fig. 22B) is positioned exactly within the guiding frame at the center of the 
screen p<3). In case where it is positioned exactly within the guiding frame, he pushes the release button 63 and other* 
wise the regulates the angle of the camera so that the desired through-image is positioned exactly within the guiding 
frame at the center of the screen (X4). 

The CPU 21 examines whether the user has pushed the release button 63 (X5). In case where the release button 
10 has been pushed, the image 83 of the object displayed on the liquid crystal display 20 (Fig. 22E) is cut-out to be 
recorded in the flash mennory 14 as a photographed image (X6). 

(t can be not cutout at imaging, txit a through-image is recorded as it is and K can be cut-out later from a repro- 
duced image (that is, it can be used as a measure off imaging). 

Further, although In the aljove explanation it is supposed that the guiding frame is one for photographic imaging, it 
IS may t>e either one of mask patterns and cut-out images. Moreover the mask patterns and the cut-out images may be 
those previously registered by a maker, etc. in a recording medium such as a flash memory. 

In this way, since a guiding image can be displayed at imaging, it is possible to make the size of images alnx)st even 
and therefore it is possible to take easily photographs for certificate, for which the size is preserved. 

20 [EMBODIMENT 8] Example of contour extraction by means of a pointing device or a cursor 

Figs. 23A to 23D are diagrams for explaining an example of mask pattern production by contour extraction. 
Fig. 23 A is a reproduced recorded image, which corresponds to the reproduced innage selected by the user in the 
step W1 in the flow chart indicated in Fig. 15. 
25 Fig. 23 B corresponds to the step VV2 in the flew chart indicated in Fig. 1 5, which indicates a state where a desired 
image 91 (a bear in this example) is displayed and thereafter the contour of the face of the image 91 is traced by means 
of an input pen as 92. When designation of a domain to be extracted by means of the input pen is terminated and the 
user operates the release button 63. the CPU 21 obtains a closed region by using coordinates (xj. yj) of positions, 
through which the input pen has passed. Parts other than the dosed region 93 on image data stored in a predetermined 
30 work area are offset (W3) and thereafter the remaining data are subjected to binary coding processing (W4). 

Fig. 23C corresponds to the step W5 in the flow chart indicated in Fig. 15, in which a result obtained by processing 
the data into a binary coded pattern is displayed on the liquid crystal display 20. When the user operates the release 
kXJtton 63, supposing that a desired mask pattern (Fig. 23D) has been obtained, the pattern image 95 is registered in 
the mask pattern memory 14B as a mask pattern (W12). 
35 Although, in the present embodiment, the reproduced recorded image has been traced by means of the input pen, 
it is possit>le also to obtain the mask pattern by an almost similar operation to the above described case, even if the con- 
tour of the tace of the Image 91_ js traced by means of the cursor by operating the cursor displacamerit keys 62. 

Further although, in the present embodiment, a contour of the reproduced image has been extracted by tracing it, 
also for a through-image, it is possible to obtain a desired mask pattern by effecting the contour extraction on the t>asis 
40 of the ftow chart indicated in Fig. 7, similarly to the case of the recorded image. 

[EMBODIMENT 9] Example of pattern production based on distance designation 

When the user is looking to signboards, etc on a street, there can be objects, which he wants to patternize. 
45 The nunri>er of signboards in his field of view is not restricted to 1 , but he can see a number of signboards from the 
foreground to the Ixick in a narrow field of view. In such a case a number off signboards are in a same f iekJ of view of 
the camera, distant from each other. 

The present embodiment is an example, in which objects, which are at different distances in a same field of view, 
are imaged in one recorded image as patterning candidates and reproducing the recorded image, each of the object 
so can be subjected to a mask patterning operation. 

In case where the digital camera has an autoffbcusing mechanism, the autofocusing mechanism works so that the 
focal point of tiie camera is deckled so as to be adjusted to a point at which the user's eyes are turned (at the center of 
a finder In many cases). Denoting tiie distance between the object arxi the imaging lens 1 by a, the distance between 
the imaging lens 1 and the surfece of the CCD 2 by b and the focal length of the lens by f. 1/a -i- 1 A> = 1/f is valki. 
55 Here, if the imaging lens 1 is displaced by ± Ax (Ax < b « a), 

1/(a-Aj^ + 1/(t>f Ax) 3 1/f 
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or 1/(a+Ax) + iy(b-Ax) = 1/f 

is obtained. Since b « a, the first term of the teft number, which 1/(a+Ax), can be regarded as 1/a. Therefore 
5 1/a + 1/(b±Ax) = 1/f (Equation 1) 

is valid. 

Since displacement AX of the lens at the focused state owing to the autofocusing control is known, the distance a 
between the object and the imaging lens can be calculated by using Equation 1 . Regulating slightly the camera so that 
10 objects, which are patterning candidates in a same field of view, are at the point at which the finder is turned one after 
another, distances between different patterning carxjidates and the lens can be obtained. Even if the digital camera has 
no autofocusing mechanism, by using an imaging device provided wnth a distance measuring mechanism it is possible 
to obtain distances between different patterning candidates and the lens. 

Figs. 24A and 24B are flow charts showing examples of the pattern production operation on the basis of the dis- 
15 tance designation, in which Fig. 24A indicates an operation of the distance data storing means 11 26 at imaging, while 
Fig. 24B indicates a corresponding operation at reproduction (at patteming). 

By the method described above distances aj (j = 1 ~ n ) between different objects, which are patterning candidates 
in a same field of view, and the imaging lens 1 are obtained and stored temporarily in a distance data storing region in 
the DRAM 13 (Y1). 

20 In case where the user effects an imaging by pushing the release button 63, the process proceeds to Y3 and other 

wise the process returns to Y1 (Y2). 

In case where an imaging is effected in the step Y2, the different distance data of the image are read-out from the 

distance data storing region in the DRAM 13 at the same time as recording of the image data and stored in a distance 

column in the link table 50 (Y3). 
25 At reproduction the distance data of the image are taken-out from the link table 50 at the same time as the desired 

reproduced image is displayed (Y4) to be displayed together with the reproduced image (Y5). 

When the user operates the displacement keys 62 to designate either one of the displaced distances t>y means of 

the cursor (Y6) and pushes the release button 63 (Y7), the image of the object which is at this distance is extracted (Y8) 

to be displayed (Y9). 

30 vsmen the user pushes the mask button 66 (Y10), the displayed object image is subjected to the mask patterning 
processing such as the binary coding processing, contour extraction processing, etc. (T3 ~ T1 1 in Fig. 7} (Y1 1) and reg- 
istered as a mask pattern (Y1 2). In case where the displayed object images include, apart from the relevant patterning 
candkiate, an image of another object at the same distance, the contour extracting processing described above 
(Embodiment 6) can be effected (Y14) by operating the displacement keys 62 (Y13). 

35 In case where the contour extracting processing is effected, the images other than the pattern carxfidate are offset 
and the process proceeds to the patterning processing Y 11 of the mask (Y1 5). 

Further it is possble to obtain a plurality mask patterns from a same image k>y repeating such an_operation. More- 
over the apparatus may be so constructed that specifying a distance at displaying the through-image, an image of an 
object which is at that distance is extracted to be registered as a mask pattern. 

40 In addition, it is possible also to designate a distance at displaying a through-image to extract an image to be 
infiaged, which is at that distance, and to register it as a mask pattern. 

[EMBODIMENT 10] An example Of input of an image drawn by means of a cursor or a pointing device 

45 When the direct button 68 is pushed under the through-image synthesizing nrxxie or the recorded image synthesiz- 
ing mode, it is possible to input a trajectory (a figure obtained by connecting discontinuous designated points) drawn on 
the screen by means of a cursor or a pointing device as a pattern and to register it as a mask pattern after having sub- 
jected it to extracting processing. 

Fig. 25 is a flow chart showing an example of operation of the pattern production means 1 125 and Figs. 26A to 26Q 

so are diagrams Ibr explaining formation of a mask pattern, starting from a pattern drawn by means of a cursor. 

In case where tiie direct button 68 is pushed-down under the through-image syrrthesizing mode (Z1), the CPU 21 
controls the timing generator 12 so that supply of image data to the signal generator 16 is interrupted and displays an 
instruction message to start input of a drawn image and the cursor 310 on the liquid crystal display 20, as indicated in 
Fig. 26A (Z2). 

65 When the user operates the displacement keys 62 so as to move the cursor continuously, a trajectory 311, along 
which the cursor is displaced, is displayed (Fig. 26B) (Z3) and coordinates (xi, yj) on the trajectory are taken in a work 
area in the DRAM 13 one after another (Z4). 

At this time, when the user pushes-down the down-key 67 (-) to move the cursor, the trajectory of the cursor is not 




20 




EP0853 426A2 



displayed and cxxjrdinates of the trajectory are not taken in the work along the trajectory, superposed thereon, parts of 
the trajectory traced thereby can be erased (Fig. 26C) (Z6). 

When the user pushes-down the up-key 67 display of the trajectory of the cursor is started and taking-in of 
coofdinates of the trajectory begins again (Z7). 
5 When the release button 63 Is pushed-down (Z8) after having formed a drawn figure, which is a dosed region 

enclosed by a continuous line obtained by repeating the steps Z5 and Z6, an icon 301 for shaping a stra^ht-line-like 
continuous line into a straight line, icons 302 and 303 for giving a curve^ike continuous line smoothness, and icons for 
typical forms (e.g. a circle 304, an ellipse 305, a triangle 307, etc. ) are drawn and displayed together with the figure 
(Fig. 26E) {Z9). 

10 When the user operates the displacement keys 62 and designates a desired icon after having designated a section 
PQ {(xa. yb), (xc, yd)} on the drawn figure 300 by means of the cursor (Fig. 26E). that section is shaped into the form 
of the designated icon (Fig. 26F). In case where the user wants to shape a part of the figure into typical one, when the 
user points an icon for a desired typical form after having pointed an inner region (a point within the dosed region) of 
the drawn figure 300, the region is shaped into the typical form (Z10). 

IS After the shaping has been terminated, when the user operates the up-down keys 67, enlargement/reductk)n of the 
figure is effected. A reduced pattern 300* is indicated in Fig. 26Q (Z1 1 , Z12). When the user pushes down the release 
txjtton 63. the figure on the screen is extracted as a pattern (Z13. Z14). 

When the pattern is extracted, it is subjected to the mask patterning procession (Z15) (reler to the steps T3 ~ T1 1 
in Fig. 7) and it is registered in the mask pattern memory 14B (Z16) after the mask patterning processing has been ter- 

20 minated. 

[EMBODIMENT 1 1] An example of display of a preview image (in case where the boundary of the nfiask pattern is 
blurred) 

2S The present embodiment is an example, in which display of a preview image effected before a synthesizing 
processing is effected with a high speed, which display is e.g. a display of a registered mask pattern (U3 in Rg. 9, XI 
in Fig. 21). a display of displacement of an image for cutting-out (the step V3 in Fig. 12A, the step S3 in Fig. 1 7, the step 
S21 in Fig. 19), adisplay of a background image (T1 in Fig. 7. U1 in Fig. 9, the step V1 in Fig. 12A, the step W1 in Fig. 
15. the step SI in Fig. 17), etc. 

30 For the image synthesis, there is disposed a memory region (work area) of 1 image (1 frame) in the DRAM 13. At 
the image display image data of one image to be displayed at that time are read-out from the mennory (the mask pattern 
memory 14B for a mask pattern, the cut-out image memory 14A for a cut-out image, and the flash memory 14 for a 
recorded image) where Ihey are stored and stored in the work area. 

Here it is supposed that coordinates of the position of the left upper corner of the screen of the liqukJ crystal display 
35 20 are (0, 0). At first, data corresponding to the coordinates (0. 0) on the screen are written at a position (0, 0) in the 
work area. Then data corresponding to the coordinates (xi. yj : i = 0 n-1 . j = 0 - m-1 ) on the screen are written at 

positions (xi, yj) in the wojic area one after another and finally data of 1 image are written-Jn. 

Here the coordinates of the diiferent data wrftten in the work area are defined as foiiows fbr the sake of explanation. 

40 coordinates of mask pattern data : M (xi, yj) 
coordinates of cut-out image data : C (xi. yj) 
coordinate of t>ackground image data : B (xi, yj) 

display coordinates (coordinates of data in the work area after the synthesizing processing) : W (xi, yj) 

46 It is supposed also that the data are expressed in 8 bits. 

In this case, e.g. in case where a mask p>attern is displayed on an image, from which a cut-out image is formed, (a 
background image) fbr selecting the mask pattern, the mask pattern which is a selection candidate arKi the background 
image, which is the image, from which a cut-out image is formed, are subjeded to a synthesizing processing. Ttie coor- 
dinates of the display position of the mask pattern at this time are determined in a coordinate calculation for the image 

GO synthesizing processing as: 

W(xi. yu) - m (xi. yD x B (xi, yD + (255- M (xi, yD x C (xi, yD) /256 

where the numerical value 255 is a brightness. 
56 Further, in case where a cut-out image is produced by cutting-out a t>ackgroind image serving as an image, from 
which a cut-out image is formed, by means of a mask pattern, a cut-out carxiidate image is produced/displayed, syn- 
thesizing by combining the background image with the mask pattern while displacing the t>ackground image, when the 
user selects desired one among cut-out candidate images, he can cut-out (extract) it as a cut-out image. If a displace- 
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ment amount is represented by 8 bits in the abscissa and q bits in the ordinate, the display coordinates in this case in 
the coofdinate calculation for the image synthesizing processing area decided as follows: 

W (xi, yD = {M (xi-p. yHO X B (xi. yj) + (255 - M (xi-p. yj<|)) x C (xi-p. yj-q)} /255 

5 

Although, in the present embodiment, modules for display coordinate determination are incorporated in the pattern 
production means 1 125 (or 1225) and the cut-out Image production registration means 113 (123) for calculating the dis- 
play coordinates, in order to increase the speed of coordinate calculations at a prev^iew display for a high speed image 
display, the apparatus is so constructed that coordinate calculations are eflected by classifying modules for display 
10 coordinate determination into sey/eral cases as stated below. (I) The display coordinates of the mask pattern at selecting 
the mask pattern are calculated by using: 

W(xi,yj) = C(xi,yj) 

IS when the brightness of M (xi, yj) is lower than a certain threshold value, and 

W(xi.y]) = B(xi. yj) 

when the brightness of M (xi, yj) is higher than the certain threshold value, 
20 and this processing is effected over the whole region (i = 0 ~ n-1 , J = 0 - m-1 ) of the work area con^esponding to 1 
image. On the basis o1 tNs result, the mask pattern is displayed on the liquid crystal display 20 by using the cut-out 
image production/registration means 1 13 (123). In the present embodiment the threshold value is supposed to be 128. 

According to the above coordinate calculation rt is possible to display the mask pattern with a speed several times 
as high as a synthesizing speed at a usual image synthesizing processing (coordinate calculation speed). Further, 
2S although the boundary between the mask pattern and the background is clearer in this case, no problem is produced, 
even if the boundary is clear, because it is no synthesized display in practice, but it is a preview display. 

(J) The display coordinates of the cut-out image at selecting the cut-out image are calculated by using; 

30 W (xi. yj) = C (xi+p, yj4q) 

when the brightness of M (xl, yj) is higher than the certain threshold value, 
and this processing is effected over the whole region (i = 0-n-1,j = 0- m-1 ) of the work area corresponding to 
1 image. On the basis of this result, the cut-out image is displayed on the liquid crystal display 20 by using the 
35 through-image (recorded image) synthesizing means 114 (124). Here p and q represent displacement amounts 
(numbers of dots) in the abscissa and the ordinate, respectively. In the present errtxxfiment the threshold value is 
supposed to be 128. According to the above coordinate calculation it is possit>le to display the cut-out inr^e with 
a speed several times as high as a synthesizing speed at a usual image synthesizing processing (coordinate cal- 
culation speed). 

40 (K) For the display coordinates of a background image (a through-image or a reproduced recorded image) at dis- 
playing a background image, 

real data are written as they are. i.e. coordinates of the through-image or the reproduced recorded image 
are written as they are in the work area. That is, 

45 W(xi.yj-)=B(xi,yj) 

is valki and this processing is effected over the whole region (i = 0~n-1,J-0 - m-1) of the work area correspond- 
ing to 1 image. On the basis of this result, the background image is displayed on the liquki crystal display 20 by 
using the pattern production registration means 112 (122). the cut-out image production/iregistration means 113 
60 (1 23) , the through image synthesizing means 1 24 or the recorded image synthesizing means 124. 

(L) At selecting a sticking position, denoting displacement amounts by p dots in the abscissa and q dots in the ordi- 
nate and a displacement amount of the sticking position from the center by r and s, respectively, the display coor- 
dinates of a cut-out image is calculate, supposing 

55 W (xi, yD = C (xi4p-r, yj-q-i-s) 

when the brightness of M (xi-r. yj-p) is lower than a certain threshold value, and 
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W (xi. yj) = B (xi+p-r. yj-q+s) 

when the brightness of M (xi-r, yl^) is higher than the certain threshold value. 
This processing is effected over the whole region (i = 0 n-1 , J = 0 m-1 ) of the work area conresponding to 1 
5 image. On the basis of this result, the cut-out image is positioned at the sticking position and displayed on the liquid 

crystal display 20 by using the thorough-image (recorded image) synthesizing means 114 (124). In the present 
embodiment the threshold value is supposed to be 128. 

According to the above coordinate calculation it is possible to display the cut-out image with a speed severeU times 
10 as high as a synthesizing speed at a usual image synthesizing processing (coordinate calculation speed). 

[EI\/IBODlMENT 12] An example of automatic determination of resolution of a synthesized image 

Since the data capacity of an image is great, while the memory capacity is limited, in case where an image is stored 
IS in a memory, the image (data) is subjected to JPEG compression processing to be recorded and at reproduction a 
recorded image (data) is subjected to extension processing to be reproduced. 

At the compression processing of an image, if resolution is increased, although the ratio of compression increases, 
processing duration increase. Therefore recording speed is lowered. On the contrary, if resolution is decreased, 
although the ratio of compression decreases, processing duration decreases. Therefore recording speed is raised. 
20 A digital camera is usually so constructed that the user can select a resolution Since resolution of the cut-out Image 
or the background image of an image to be processed differs, depending on specification of that product, choice of res- 
olution of the synthesized image is diversified. 

In the present embodiment, in order to record rapidly the synthesizing image, it is supposed that resolution is 
adjusted to the lowest value and comparing an image recorded in the cut-out image mennory 14A and an image 
26 recorded in a background image buffer (DRAM 13) in resolution by means of resolution automatically determining 
means, resolution of a synthesized image is determined to be the lower resotutton. A synthesized image is compressed 
by the compression/extension circuit 1 5 on the basis of the resolution thus determined to be recorded in the flash mem- 
ory 14. 

Further the coordinate calculation at the synthesizing processing combining the cut-out image with the background 
30 image differs from that used for the display coordinate in the case of the preview display That is. denoting displacement 
amounts by p dots in the ak}scissa arxl q dots in the ordinate arxi displacement amounts of the sticking position from 
the center by r and s, respectively, the coordinates at the synthesizing processing are calculated, supposing 

W (xi. yj) = M (xi-r, yj-s) x B (xi, yj) + (255 - M (xi-r. yj-s)) x C (xi+p-r, yj+q-s)) / 255. 

35 

and the synthesis is executed by effecting this processing over the whole region (i = 0 - n-1 , j s 0 - m-1 ) of the work 
area corresponding to 1 image. 

As explained in the above, by the imaging method and the image processing method according to the present 
invention, the user can extract a mask pattern from a through image and an image recorded in a recording medium to 
40 register it in the recording medium and produce a cut-out image by using the through image and the image recorded in 
the recorded medium, which is cut-out and registered in the recording medium, in a simple manner. Further, since it is 
possit)te to produce a synthesized image by combining the cut-out image and a taken image or a recorded image to 
record it, a combined processed image can be simply obtained under a free conception of the user. 

Moreover, since a guiding frame can be displayed at imaging, it is possible to make the size of images almost even 
45 or to image in a simple manner a photograph for certifk»te, etc., for wNch the size is prescrlt>ed. 

Although various embodiments of the present invention have been explained in the above, the present invention is 
not restricted to the embodiment described above, txjt it is a matter of course that various modified realizations are pos- 
sible. 

60 Claims 

1 . An image pickup apparatus (100) including inr^ge pickup means (41) for imaging a suk^ect to output an image and 
image memory means (13. 14) for storing the image outputted from said image pickup means (41), saki apparatus 
being characterized by enrployment of: 

55 

pattern memory means (148) for storing a pattern, dipped image producing means (21) for clipping a conre- 
sponding part of the image stored in said image memory means (13, 14) with the pattern stored in sakI pattern 
memory means (14B) to produce a clipped image, and 
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image composing means (21) for composing a desired image said clipped image produced by said clipped 
image producing means (21). 

2. An image pickup apparatus (100) according to Claim 1 wherein said pattern menrwry means (14B) stores a plurality 
5 of patterns, comprising; 

pattern select means (24) for selecting desired one of said patterns, wherein said clipped image producing 
means (2 1 ) clips a con'esponding part of the image stored in said image memory means (13.14) with the pat- 
tern selected by said pattern select means (24) to produce a clipped Image. 

10 

3. An image pickupt apparatus (1 00) according to Claim 1 comprising pattern producing means (21 ) for producing the 
pattern from the image stored in said image memory means (13, 14), wherein said pattern memory nDeans (14B) 
stores said pattern produced by said pattern producing means (21). 

IS 4. An image pickup apparatus (100) according to claim 3 wherein said pattern producing means (21) produces the 
pattern by extracting a profile of the image. 

5. An image pickup apparatus (1 00) acoorcfing to Claim 3 comprising appointing (62) means for appointing an innaging 
distance and wherein said pattern producing means (21) produces the pattern by extracting an image part corre- 

20 spending to the imaging distance in the image appointed by said appointing means (62). 

6. An image pickup apparatus (100) according to Claim 3 wherein saki pattern producing means (21) produces the 
pattern tay binary-coding the image. 

2S 7. An image pickup apparatus (100) according to Claim 3 wherein sakJ patten producing means (21) produces the 
pattern by extracting a luminance part of the image. 

8. An image pickup apparatus (100) according to Claim 1 comprising drawing data inputting means (24) for inputting 
a drawing data and pattern producing means (21) for producing saki pattern from the drawing data, wherein said 

30 pattern memory means (14B) stores said pattern produced by saki pattern producing means (21). 

9. An image pickup apparatus (100) according to Claim 8 wherein saki drawing data inputting means (24) is a cursor 
or a pointing device for drawing. 

35 10. An image pickup apparatus (100) according to Claim 1 comprising: 

displaying means (20) for displaying a moving image outputted from ssa6 image pickup means (41), display 
control means (21) for making said pattern stored in pattern memory means (14B) compose on saki moving 
image displayed on sakJ displaying means (20) and display a composed one thereon, 
40 key inputting means (63), and 

memory control means (21) for making the image store in said image memory means (14) in a case that said 
key inputting means is operated while the moving image and the pattern is displayed on said displaying means 
(20) by saki display control means (21). 

45 1 1 . An Image pickup apparatus (1 00) according to Claim 10 wherein saki clipped image producing means (21) clips a 
corresponding part of the Image with the pattern displayed on said displaying means (20) to produce the dipped 
image and said memory control means (21) makes said clipped image produced by saki clipped innage producing 
means (21) store in saki image memory means (14). 

60 12. An image pickup apparatus (100) according to Claim 10 comprising reproducing means (20) for reproducing the 
image stored in said image memory means, wherein said clipped image producing means (21) clipe a correspond- 
ing part of the image reproduced by saki reproducing means (20) with the pattern stored in said pattern menrK>ry 
means (14B). 

ss 1 3. An image pickup apparatus (1 00) according to Claim 1 comprising: 

displaying means (20) for displaying a moving image outputted from said image pickup means (41), 
displaying control means (21) for making the clipped innage produced by said clipped image producing means 
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(21) composite on the moving image displayed on said displaying means (20) and display a composed one 
thereon, 

key inputting means (63), and 

memory control means (21) for making the image store in said image memory means (14) in a case that said 
5 key inputting means (63) is operated while the moving image and the dipped image is displayed on said dis- 

playing means (20) by said display control means (21). 

1 4. An Image pickup apparatus (1 00) according to Claim 1 3 wherein sakl memory control means (2 1 ) makes the image 
and the clipped image displayed on said displaying means (20) store In said image memory means (14). 

10 

15. An image pickup apparatus (ICQ) according to Claim 13 comprising: 

displacement appointing means (62) for appointing displacement of the clipped image displayed on the moving 
image t>y said display control means (21) to a desired p06itk>n within a display plane thereof. 

IS 

16. An image pickup apparatus (100) according to Claim 1 comprising: 

displaying means (20) for displaying the inriage stored in said image mennory means (13,14), 
display control means for making the pattern stored in sakl pattern memory means (14B) display on the image 
20 displayed on said displaying means (20), 

displacing means (21) for displacing said pattern displayed on said image by eaM display control means to a 
desired position within a displaying plane of sakJ display means (20), wherein said clipped image producing 
means (21) clips a corresponding part of swd image with said pattern displaced by said displacing means (21). 

25 17. An Image processing method comprising: 

first step clipping a corresponding part of an image wnth a stored pattern, and 

a second step composing a desired image the clipped image to produce a composed image. 

30 18. An irr^ge processing method according to Claim 1 7 wherein said first step includes a step selecting a desired one 
from a plurality of stored patterns to clip a corresponding part of said image with the selected pattern. 

1 9. An image processing method according to Claim 1 7 wherein saki first step include a step producing a pattern from 
said image to dip a corresponding part of said image with the produced pattern. 

35 

20. An image processing method according to Claim 17 wherein said first step includes a step inputting drawing data 
and a step producing said pattern from tha inputted drawing data to clip a corresporKling part of sakJ image with 
the produced pattern. 

40 



45 



60 



65 



25 



EP0853 426A2 



< 
> 



o 
o 

liJ 
9 

> 

Q 



is 

UJ 

CO 



0> 



a 
o 
in 

< 

CC 

< 

o 

9 



CC 



o 



-CM 



i 



in. 

CM 



QflC 

SO 



i 



CM. 



So 



(A COS 
liJZO 
OCUJOC 

o 



^ S3 



< 

CC 



CM 



3 
O 



o 

CC 



RE 

Co 



111 



CC 
z ^ 

1-Z 
UJ 
O 



O 
O 

QQ 

9 



^ 1 ^ 



CO 



CM 



o 
s 

UJ 

s 

X 
CO 

< 



K >- 
3UJCC 
0C30 

^Suj 



< 

CE 
Q 



CO 



(5 



m 




26 



EP0853 426 A2 



61 



63 

4: 



64 

± 



FRAME 



<] 



A 



V 



RELEAS 



4- 



REPRODUCTION 



RECORD 



MASK 



DIRECT 



65 

66 
68 



67 UP-DOWN 
KEYS 

62 DISPLACEMENT 
KEYS 



24 KEY INPUT 
SECTION 



FIG.2 



27 




28 



EP0 853 426A2 



110 



(21)— 



MODE 
JUDGING 
MEANS 



120 



GUIDING FRAME 
DISPLAY MEANS 



^130 



(14) 



111 



PROCESSING 
METHOD 
JUDGING 
MEANS 



PATTERN 
PRODUCTION/ 
REGISTRATION 
MEANS 



.112 



CUT-OUT IMAGE 
PRODUCTION/ 
REGISTRATION 
MEANS 



.113 



115 



114 



THROUGH-IMAGE 
SYNTHESIZING 
MEANS 



; PROCESSING ! 
— : INFORMATION ^ 
; GIVING MEANS \ 



121 



PROCESSING 
METHOD 
JUDGING 
MEANS 



PATTERN 
PRODUCTION/ 
REGISTRATION 
MEANS 



.122 



CUT-OUT IMAGE 
PRODUCTION/ 
REGISTRATION 
MEANS 



.123 



125 



124 



RECORDED IMAGE 
SYNTHESIZING 
MEANS 



, PROCESSING ; 
INFORMATION |- 
GIVING MEANS I 



FIG.4A 



-<14B) 



.(14A) 



(14) 



(14B) 



(14A) 



(14) 



■1125 r 



112 



; PATTERN 
: PRODUCTION 
MEANS 



BINARY 
CODED 
PATTERN 
PRODUCTION 
MEANS 



.^1121 



! DISTANCE 

DATA 
; STORING 
MEANS 



LUMINANCE 
COMPONENT 
PROCESSING 



.1122 



122 



,-1225 



PATTERN 

: PRODUCTION ; 

MEANS 



BINARY 
CODED 
PATTERN 
PRODUCTION 
MEANS 



LUMINANCE 
COMPONENT 
PROCESSING 



.1221 



1222 



1126 



FIG.4B 



FIG.4C 



29 




EP0 853 426 A2 



FIG.5A 



IMAGE 
NUMBER 


IMAGE 
NUMBER 


CUT-OUT 

IMAGE 
NUMBER 


















^^^^^^ 






* 1 







so 



FIG.5B 



IMAGE 
NUMBER 


CUT-OUT 

IMAGE 
NUMBER 

































513-n 



CUT-OUT 

IMAGE 
NUMBER 



FIG.5C 



r 



513 



r 



514 



.^512 



50 



515 



CUT-OUT 
IMAGE 
NUMBER 


MASK 
PATTERN 
NUMBER 


IMAGE 
NUMBER 


SYNTHESIS 
POSITION 







































30 




EP0 853 426 A2 





31 



EP0 853 426 A2 



START OF PATTERN 
PRODUCTION/REGISTER 



0 



DISPLAY OBJECT 
TO BE IMAGED 



r 



T1 



NO. 



IS MASK BUTTON 
PUSHED-DOWN ? 



< 



YES 



IS UP-DOWN 
KEY OPERATED? 



NO 



NO 




TRANSFORMATION INTO 
BINARY CODED PATTERN 






DISPLAY OF 
BINARY CODED PATTERN 




. /-TO 




IS RELEASE BUTTON 
PUSHED-DOWN ? 



YES 



> 



T7 



IS LUMINANCE 
COMPONENT MODIFICATION 
OPERATION EFFECTED 



YES 

1/ 



NO 



T8 



LUMINANCE 
MODIFICATION 



PATTERNIZE LUMINANCE 
COMPONENT PROCESSED 
IMAGE 



r 



I 



T9 



DISPLAY LUMINANCE 
COMPONENT PROCESSED 
PATTERN 



r 



T10 



NO 




IS RELEASE BUTTON 
PUSHED-DOWN ? 



YES 



REGISTER IMAGE 
AS MASK PATTERN 



T12 



FIG-7 



32 



EP0 853 426A2 




33 



EP0 853 426A2 



C 



START OF CUT-OUT IMAGE 
PRODUCTION/REGISTER 



:) 



DISPLAY OBJECT 
IMAGE 



NO^ 



r 



U1 



IS MASK BUTTON 
PUSHED-DOWN 



U2 



YES 



U3 



MASK PATTERN IS COMBINED 

WITH OBJECT IMAGE 
SUPERPOSED THEREON TO BE 
DISPLAYED 



NC^ 



U4 



IS IT DESIRED 
PATTERN ? 



YES 



< 



> 



IS CORRECTION OF 
POSITION NECESSARY ? 



NO 



< 



YES 



ANGLE 
REGULATION 



NO 



U6 



IS RELEASE BUTTON 
PUSHED-DOWN ? 



> 



YES 




SUPERPOSED PART 
IS CUT-OUT AND 
REGISTERED AS CUTOUT IMAGE 







r 



U7 



FIG.9 



34 



EP0 853 426 A2 





36 



EP 0 853 426 A2 



C 



START OF THROUGH 
IMAGE SYNTHESIS 



DISPLAY OBJECT 
IMAGE 



r 



VI 



IS FRAME BUTTON 
PUSHED-DOWN ? 



YES 




V2 



DISPLAY SYNTHESIZED 
IMAGE WITH CUT-OUT IMAGE 



V4 



i: 



IS CUTOUT 
IMAGE OK ? 



YES 



> 



NO 



IS IT NECESSARY 
TO CORRECT POSITION ? 



< 



NO 



SHOULD IT BE 
RECORDED ? 



NO 



YES 



IS RELEASE BUTTON 
PUSHED-DOWN ? 



YES 



RECORD SYNTHESIZED 
IMAGE 



^YES ^ 


ANGLE 




REGULATION 



V5 



c 



END 



D FIG.12A 



<IS IT I 
CORR 
— T 




NECESSARY TO \YES 
CORRECT POSITION 7 




OPERATION OF 
DISPLACEMENT KEYS 



r 



V5' 



NO 



DISPLACEMENT OF 
CUT-OUT IMAGE FOR 
STICKING PROCESSING 



V 



V5" 



FIG.12B 



37 



EP0 853 426 A2 





38 



EP0 853 426 A2 




39 



EP0853 426 A2 



c 



START OF PATTERN 
PRODUCTION/REGISTRATION 



REPRODUCE 
RECORDED IMAGE 



NO> 



W1 



W2 



IS MASK BUTTON 
OPERATED ? 



< 



YES 



IS UP-DOWN 
KEYS OPERATED ? 



NO 




W7 



W4 



NO 




TRANSFORM 
BINARY COD 


lATION INTO 
ED PATTERN 






DISPLAY OF 
BINARY CODED PATTERN 




^W6 



IS RELEASE BUTTON 
PUSHED-DOWN ? 



YES 



> 



IS LUMINANCE 
COMPONENT MODIRCATION 
OPERATION EFFECTED 



NO 



W8 



LUMINANCE 
MODIHCATION 



TRANSFORMATION INTO 
LUMINANCE COMPONENT 
PROCESSED PATTERN 



W9 



DISPLAY OF LUMINANCE 
COMPONENT PROCESSED 
PATTERN 



r 



W10 



NO X 



IS RELEASE BUTTON 
PUSHED-DOWN 



YES 



V/-W11 

ON X 



REGISTRATION 
AS MASK PATTERN 



W12 



C 



END 



FIG.15 



40 



EP 0 853 426 A2 



BEST AVAILABLE 



172 171 /20 



172 



FIG.16A 



FiG.16B 



20 



173 



FIG.16C 



20 



173* 



FIG.16D 



41 



EP0853 426 A2 



C 



START OF 
STICKING SYNTHESIS 



REPRODUCE AND DISPLAY 
RECORDED IMAGE 



r 



TURN-ON 
FRAME BUTTON 



r 



S1 



S2 



S3 



DISPLAY CUT-OUT IMAGE 



NO^ 



S4 



IS SELECTION 
TERMINATED ? 



YES 



<ISIT 
CORI 



> 



S5 



NECESSARY TO 
CORRECT POSITION ? 



YES 



NO 



S6 



OPERATE 
DISPLACEMENT 
KEYS 



S8 



SHOULD IT BE 
RECORDED ? 



> 



YES 




TURN-ON 
RELEASE BUTTON ? 






RECORDING PROCESSING 



S7 



DISPLACE 
REPRODUCED 
IMAGE 



r 



S9 



v 



S10 



END ^ 

FIG.17 



42 



EP0 853 426 A2 




axz 




43 



EP0 853 426 A2 



c 



START OF PRODUCTION 
OF CUT-OUT IMAGE 



3 



REPRODUCE/DISPLAY 
RECORDED IMAGE 



r 



S21 



TURN-ON 
MASK BUTTON 



f 



S22 



MASK PATTERN DISPLAY 



S23 



MASK PATTERN SELECTION 



S24 



DISPLAY MASK PATTERN 
ON REPRODUCED IMAGE 



V 



<ISIT 
CORI 



S25 



S26 



NECESSARY TO 
CORRECT POSITION 



YES 



NO 



r 



S29 



TURN-ON 
RELEASE BUTTON 



JL 



S27 



OPERATE 
DISPLACEMENT 
KEYS 



S30 



RECORD IT AS 
CUT-OUT IMAGE 



S28 



DISPLACE 
MASK PATTERN 
POSITION 



FIG.19 



44 



EP 0 853 426 A2 



ujotOO 
Q»-5 




ill 
gill 



CD 

o 

CM 

d 



CM 



C5 

CCXUJi^ w 



Q 

UJ^UJ 





is* S 

ii§ <^ 

<Ot: LI 

€0 



Q 
O 

CM 

d 



45 



EP 0 853 426 A2 



C 



START OF GUIDING 
FRAME DISPLAY 



3 



I 

DISPLAY GUIDING U 
FRAME PATTERN 



IS PATTERN OK ? 



YES 



< 



X2 



X3 



IS IMAGE 
POSITION OK ? 



> 



YES 



NO 



X5 



^ RECORDED ? ^ 


^X6 




YES 


RECORD IT AS 
RECORDED IMAGE 


1 









C gND ) 



Jl 



X4 



ANGLE 
REGULATION 



FIG .21 



46 



EP 0 853 426 A2 




EP0 853 426 A2 




48 



EP0 853 426 A2 



C START : ) 





DISPLAY 
REPRODUCED IMAGE 








DISPLAY 
DIFFERENT DISTANCES 












DESIGNATE DISTANCE 


NO 


RELEASE BUTTON^ 



-Y4 



Y5 



.Y6 



C START ) 



STORE 
TEMPORARILY 
DISTANCE DATA 




Y1 



OPERATED 
YES* 



STORE IMAGE AND 
DISTANCE DATA 







EXTRACT RELEVANT 
OBJECT IMAGE 






DISPLAY E 
OBJEC 


iXTRACTED 
T IMAGE 



Y9 



FIG.24A 



Y10 



Y13 




NO 



Y14 



Y11 



MASK PATTERNING 
PROCESSING 



Y12 



REGISTER 
MASK PATTERN 



CONTOUR PORTION 
EXTRACTING 
PROCESSING 



Y15 



OFFSET IMAGES 
OTHER THAN 
PATTERNING CANDIDATE 



FIG.24B 



49 



EP 0 853 426 A2 




TO OTHER PATTERN 
EXTRACTING PROCESSING 



DISPLAY INSTRUCTION 
MESSAGE AND CURSOR 



DRAWING BY CURSOR 
DISPLACEMENT 



TAKE-IN ONLY COORDINATES } 




ERASE UNNECESSARY 
PARTS OF TRAJECTORY 




ENLARGING/REDUCING 
PROCESSING 



Z13 




Z14-i 
Z15- 

Z16-H 



EXTRACT PATTERN 



MASK PATTERNING 
PROCESSING 



FIG.25 



REGISTER MASK 
PATTERN 



50 



EP0 853 426 A2 



DRAW 




FIG.26A FIG.26B 




FIG.26C FIG.26D 




FIG.26E FIG.26F 




51 



